-- Logs begin at Sat 2022-06-18 04:08:00 UTC, end at Sat 2022-06-18 05:35:40 UTC. -- Jun 18 04:57:21.562850 dv-cinder-836996-35-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jun 18 04:57:21.571595 dv-cinder-836996-35-iscsi devstack@keystone.service[80454]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jun 18 04:57:21.573681 dv-cinder-836996-35-iscsi devstack@keystone.service[80454]: *** Starting uWSGI 2.0.18-debian (64bit) on [Sat Jun 18 04:57:21 2022] *** Jun 18 04:57:21.573681 dv-cinder-836996-35-iscsi devstack@keystone.service[80454]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jun 18 04:57:21.573681 dv-cinder-836996-35-iscsi devstack@keystone.service[80454]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jun 18 04:57:21.573681 dv-cinder-836996-35-iscsi devstack@keystone.service[80454]: nodename: dv-cinder-836996-35-iscsi Jun 18 04:57:21.573681 dv-cinder-836996-35-iscsi devstack@keystone.service[80454]: machine: x86_64 Jun 18 04:57:21.573681 dv-cinder-836996-35-iscsi devstack@keystone.service[80454]: clock source: unix Jun 18 04:57:21.573681 dv-cinder-836996-35-iscsi devstack@keystone.service[80454]: pcre jit disabled Jun 18 04:57:21.573681 dv-cinder-836996-35-iscsi devstack@keystone.service[80454]: detected number of CPU cores: 4 Jun 18 04:57:21.573681 dv-cinder-836996-35-iscsi devstack@keystone.service[80454]: current working directory: / Jun 18 04:57:21.573681 dv-cinder-836996-35-iscsi devstack@keystone.service[80454]: detected binary path: /usr/bin/uwsgi-core Jun 18 04:57:21.573681 dv-cinder-836996-35-iscsi devstack@keystone.service[80454]: your processes number limit is 47680 Jun 18 04:57:21.574414 dv-cinder-836996-35-iscsi devstack@keystone.service[80454]: your memory page size is 4096 bytes Jun 18 04:57:21.574414 dv-cinder-836996-35-iscsi devstack@keystone.service[80454]: detected max file descriptor number: 2048 Jun 18 04:57:21.574414 dv-cinder-836996-35-iscsi devstack@keystone.service[80454]: lock engine: pthread robust mutexes Jun 18 04:57:21.574414 dv-cinder-836996-35-iscsi devstack@keystone.service[80454]: thunder lock: enabled Jun 18 04:57:21.574414 dv-cinder-836996-35-iscsi devstack@keystone.service[80454]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jun 18 04:57:21.574414 dv-cinder-836996-35-iscsi devstack@keystone.service[80454]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Jun 18 04:57:21.618596 dv-cinder-836996-35-iscsi devstack@keystone.service[80454]: Python main interpreter initialized at 0x556e363059c0 Jun 18 04:57:21.618596 dv-cinder-836996-35-iscsi devstack@keystone.service[80454]: python threads support enabled Jun 18 04:57:21.618596 dv-cinder-836996-35-iscsi devstack@keystone.service[80454]: your server socket listen backlog is limited to 100 connections Jun 18 04:57:21.618596 dv-cinder-836996-35-iscsi devstack@keystone.service[80454]: your mercy for graceful operations on workers is 90 seconds Jun 18 04:57:21.618955 dv-cinder-836996-35-iscsi devstack@keystone.service[80454]: mapped 403077 bytes (393 KB) for 2 cores Jun 18 04:57:21.619038 dv-cinder-836996-35-iscsi devstack@keystone.service[80454]: *** Operational MODE: preforking *** Jun 18 04:57:21.619281 dv-cinder-836996-35-iscsi devstack@keystone.service[80454]: *** uWSGI is running in multiple interpreter mode *** Jun 18 04:57:21.619281 dv-cinder-836996-35-iscsi devstack@keystone.service[80454]: spawned uWSGI master process (pid: 80454) Jun 18 04:57:21.619482 dv-cinder-836996-35-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jun 18 04:57:21.621449 dv-cinder-836996-35-iscsi devstack@keystone.service[80454]: spawned uWSGI worker 1 (pid: 80468, cores: 1) Jun 18 04:57:21.621449 dv-cinder-836996-35-iscsi devstack@keystone.service[80454]: spawned uWSGI worker 2 (pid: 80469, cores: 1) Jun 18 04:57:21.621704 dv-cinder-836996-35-iscsi devstack@keystone.service[80454]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jun 18 04:57:27.124914 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 18 04:57:27.124914 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: from cryptography.utils import int_from_bytes Jun 18 04:57:27.125730 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 18 04:57:27.125730 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: from cryptography.utils import int_from_bytes Jun 18 04:57:27.127367 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 18 04:57:27.127367 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: from cryptography.utils import int_from_bytes Jun 18 04:57:27.127819 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 18 04:57:27.127819 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: from cryptography.utils import int_from_bytes Jun 18 04:57:27.174042 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 18 04:57:27.174236 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 18 04:57:27.174431 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] command line args: [] {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 18 04:57:27.174706 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 18 04:57:27.174791 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 18 04:57:27.174871 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] ================================================================================ {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 18 04:57:27.174948 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 18 04:57:27.175128 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] command line args: [] {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 18 04:57:27.175208 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] admin_token = **** {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 18 04:57:27.175373 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] config_dir = [] {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 18 04:57:27.175456 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 18 04:57:27.176075 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 18 04:57:27.176162 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] ================================================================================ {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 18 04:57:27.176245 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] config_source = [] {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 18 04:57:27.176427 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] debug = True {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 18 04:57:27.176626 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 18 04:57:27.176915 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] admin_token = **** {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 18 04:57:27.177016 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 18 04:57:27.177123 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] config_dir = [] {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 18 04:57:27.177228 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 18 04:57:27.177330 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 18 04:57:27.177330 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] config_source = [] {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 18 04:57:27.177596 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 18 04:57:27.177702 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] debug = True {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 18 04:57:27.177810 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 18 04:57:27.177908 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 18 04:57:27.178040 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] list_limit = None {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 18 04:57:27.178143 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 18 04:57:27.178239 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] log_config_append = None {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 18 04:57:27.178346 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 18 04:57:27.178448 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 18 04:57:27.178551 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 18 04:57:27.178653 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] log_dir = None {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 18 04:57:27.178653 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] log_file = None {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 18 04:57:27.178843 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 18 04:57:27.178974 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 18 04:57:27.179056 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] list_limit = None {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 18 04:57:27.179056 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] log_config_append = None {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 18 04:57:27.179245 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 18 04:57:27.179368 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 18 04:57:27.179444 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 18 04:57:27.179568 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] log_dir = None {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 18 04:57:27.179649 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 18 04:57:27.179723 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] log_file = None {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 18 04:57:27.179844 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 18 04:57:27.179926 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 18 04:57:27.180041 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 18 04:57:27.180125 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 18 04:57:27.180197 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 18 04:57:27.180312 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 18 04:57:27.180393 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 18 04:57:27.180464 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 18 04:57:27.180538 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 18 04:57:27.180619 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 18 04:57:27.180696 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 18 04:57:27.180767 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 18 04:57:27.180897 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 18 04:57:27.180973 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 18 04:57:27.181124 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 18 04:57:27.181207 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 18 04:57:27.181372 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 18 04:57:27.181482 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 18 04:57:27.181639 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 18 04:57:27.181722 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 18 04:57:27.181873 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 18 04:57:27.181965 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 18 04:57:27.182102 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 18 04:57:27.182345 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 18 04:57:27.182426 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.23/identity {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 18 04:57:27.182557 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] publish_errors = False {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 18 04:57:27.182671 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 18 04:57:27.182789 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 18 04:57:27.182903 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 18 04:57:27.183081 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 18 04:57:27.183194 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 18 04:57:27.183307 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.23/identity {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 18 04:57:27.183426 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 18 04:57:27.183553 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] publish_errors = False {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 18 04:57:27.183668 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 18 04:57:27.183780 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 18 04:57:27.183892 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] standard_threads = False {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 18 04:57:27.184030 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 18 04:57:27.184168 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 18 04:57:27.184291 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 18 04:57:27.184406 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 18 04:57:27.184569 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 18 04:57:27.184705 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 18 04:57:27.184824 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 18 04:57:27.184936 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] use_journal = False {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 18 04:57:27.184936 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] use_json = False {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 18 04:57:27.185149 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] standard_threads = False {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 18 04:57:27.185261 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] use_stderr = False {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 18 04:57:27.185374 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 18 04:57:27.185521 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] use_syslog = False {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 18 04:57:27.185646 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 18 04:57:27.185767 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 18 04:57:27.185880 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 18 04:57:27.185880 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] use_journal = False {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 18 04:57:27.186104 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.186218 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] use_json = False {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 18 04:57:27.186403 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.186525 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] use_stderr = False {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 18 04:57:27.186638 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.186751 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] use_syslog = False {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 18 04:57:27.186751 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 18 04:57:27.186962 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.187083 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.187204 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.187317 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.187430 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.187430 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.187648 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.187770 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.187890 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.188010 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.188123 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.188123 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.188341 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.188462 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.188576 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.188688 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.188800 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.188910 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.189023 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.189023 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.189530 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.189530 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.189753 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.189882 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.190010 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.190126 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.190240 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.190354 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.190467 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.190580 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.190702 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.190817 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.190932 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.191049 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.191799 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.191960 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.192088 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.192227 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.192406 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.192535 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.192658 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.192773 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.192961 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.192961 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.192961 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.192961 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.193442 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.193561 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.193689 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.193805 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.193925 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.194041 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.194160 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.194275 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.194397 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.194517 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.194631 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.194750 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.194864 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.194983 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.195107 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.195222 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.195341 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.195491 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.195614 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.195729 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.195851 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.195949 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.196051 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.196149 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.196244 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.196345 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.196450 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.196553 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.196649 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.196744 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.196879 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.197004 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.197128 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.197245 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.197349 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.197489 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.197608 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.197734 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.197850 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.197954 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.198072 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.198189 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.198306 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.198429 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.198544 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.198659 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.198781 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.198902 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.199017 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.199131 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.199252 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.199376 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.199489 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.199603 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.199725 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.199842 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.199956 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.200069 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.200185 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.200300 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.200404 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.200517 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.200631 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.200753 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.200875 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.200994 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.201110 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.201225 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.201348 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.201507 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.201633 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.201762 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.201879 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.201984 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.202087 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.202212 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.202328 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.202450 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.202565 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.202683 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.202796 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.202919 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.203036 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.203151 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.203279 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.203396 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.203512 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.203635 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.203750 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.203864 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.203984 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.204099 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.204213 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.204399 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.204518 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.204633 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.204747 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.204861 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.204975 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.205101 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.205217 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.205332 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.205462 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.205578 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.205693 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.205824 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.205953 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.206070 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.206184 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.206298 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.206412 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.206526 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.206641 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.206772 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.206892 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.207030 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.207194 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.207314 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.207439 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.207555 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.207677 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.207793 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.207908 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.208022 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.208147 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.208263 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.208367 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.208464 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.208464 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.208685 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.208809 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.208925 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.209039 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.209152 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.209266 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.209396 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.209525 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.209643 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.209758 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.209872 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.209986 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.210100 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.210221 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.210337 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.210451 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.210564 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.210678 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.210800 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.210927 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.211044 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.211158 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.211271 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.211385 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] auth.external = None {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.211505 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.211628 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.211750 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.211867 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.211982 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.212096 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.212210 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.212332 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] auth.password = None {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.212448 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.212569 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] auth.token = None {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.212683 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.212824 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.212959 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.213082 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.213197 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] auth.external = None {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.213321 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.213463 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.213581 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.213696 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.213811 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.213927 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.214054 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.214172 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] auth.password = None {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.214296 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.214413 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] auth.token = None {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.214529 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] credential.caching = True {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.214644 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.214772 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.214889 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.215006 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.215121 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.215236 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.215351 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.215474 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.215591 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.215763 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.215883 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.216004 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.216130 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.216247 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.216362 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] credential.caching = True {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.216477 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.216600 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.216717 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.216841 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.216958 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.217075 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.217190 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.217311 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.217441 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.217571 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.217571 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.217801 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.217925 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.218042 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.218158 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.218349 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.218469 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.218585 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] federation.caching = True {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.218700 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.218848 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.218848 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.219075 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.219193 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.219308 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.219423 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.219537 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.219652 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.219775 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.219892 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.220007 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.220128 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.220245 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] federation.caching = True {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.220360 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.220483 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.220600 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.220716 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.220836 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.220951 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.221066 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.221193 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.221321 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] identity.caching = True {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.221499 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.221499 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.221720 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.221720 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.221945 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.221945 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.222163 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.222163 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.222395 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.222526 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.222651 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.222756 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.222913 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.223013 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.223577 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] identity.caching = True {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.223696 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.223822 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.223940 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.224062 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.224179 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.224294 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.224411 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.224535 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.224652 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.224767 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.224888 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.225005 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.225129 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.225254 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.225371 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.225507 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.225622 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.225738 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.225870 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.226001 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.226119 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.226235 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.226350 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.226465 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.226590 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.226699 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.226813 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.226936 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.227052 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.227168 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.227291 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.227409 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.227525 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.227641 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.227756 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.227871 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.227995 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.228111 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.228227 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.228343 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.228464 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.228580 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.228704 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.228822 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.228939 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.229054 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.229169 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.229284 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.229425 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.229545 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.229662 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.229778 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.229899 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.230015 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.230138 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.230256 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.230371 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.230487 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.230602 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.230724 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.230849 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.230966 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.231321 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.231449 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.231632 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.231751 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.231875 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.231993 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.232109 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.232224 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.232340 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.232464 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.232590 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.232708 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.232833 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.232954 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.233070 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.233185 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.233308 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.233486 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.233604 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.233720 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.233836 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.233951 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.234079 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.234197 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.234312 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] ldap.user = None {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.234465 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.234581 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.234695 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.234819 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.234937 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.235052 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.235167 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.235282 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.235397 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.235521 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.235637 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.235753 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.235868 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.235984 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.236099 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] ldap.user = None {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.236222 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.236323 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.236421 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.236518 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.236616 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.236713 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.236818 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.236917 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.237015 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.237112 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.237209 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.237305 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.237431 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.237537 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.237636 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.237743 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.237861 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.238465 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.238465 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.238706 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.238822 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.238943 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.239059 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.239174 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.239297 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.239414 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.239528 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.239643 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.239643 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.239860 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.239983 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.240100 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.240222 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.240338 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.240338 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.240562 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.240679 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.240779 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.240876 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.240972 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.240972 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.240972 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.241248 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.241346 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.241346 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.241588 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.241703 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.241897 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.242018 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.242135 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.242251 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.242366 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.242481 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.242605 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.242721 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.242836 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.242951 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] resource.caching = True {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.243067 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.243182 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.243182 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.243439 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.243556 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.243669 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.243767 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.243864 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.243973 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.244073 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.244170 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.244267 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.244364 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.244460 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.244565 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.244664 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.244761 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.244857 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.244954 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] role.caching = True {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.245051 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] resource.caching = True {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.245156 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] role.driver = sql {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.245261 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.245360 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.245489 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.245590 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.245687 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.245796 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.245905 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.246003 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.246333 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.246451 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.246566 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.246690 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.246809 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.246924 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.247039 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.247153 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.247267 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.247390 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.247508 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] role.caching = True {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.247622 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.247736 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] role.driver = sql {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.247850 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.247965 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.248095 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.248095 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.248316 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.248432 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.248548 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.248548 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.248774 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.248892 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.249007 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.249105 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.249210 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.249317 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.249474 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.249580 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.249678 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.249793 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.249908 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.250036 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.250153 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.250268 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.250382 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.250496 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.250611 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.250734 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.250851 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.250965 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.251080 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.251080 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.251295 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.251422 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.251539 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.251654 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.251768 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.251882 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.251996 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.252118 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.252235 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.252349 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.252463 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.252578 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.252692 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.252816 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.252933 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.253047 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] token.caching = True {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.253162 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.253276 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.253408 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.253540 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.254296 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.254422 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.254539 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.254655 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.254769 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.254895 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.255013 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.255129 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.255129 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.255346 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.255461 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.255585 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.255703 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.255819 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.255936 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.255936 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.255936 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.256262 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.256380 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.256495 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.256611 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.256726 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] token.caching = True {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.256841 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.256966 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.257083 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.257200 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.257315 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.257482 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.257600 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] database.connection = **** {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.257733 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.257863 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.257982 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.258097 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.258212 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.258327 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.258443 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.258567 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.258684 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.258800 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.258915 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.259032 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.259147 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.259269 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.259386 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.259502 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.259616 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.259731 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.259846 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.259969 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.260087 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.260203 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.260317 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.260433 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.260547 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.260767 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.260892 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.261008 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.261125 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] database.connection = **** {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.261250 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.261250 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.261572 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.261698 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.261816 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.261933 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.262049 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.262166 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80468) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 18 04:57:27.262283 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.262283 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.262283 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.262283 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.262283 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.262283 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.262283 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.262283 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.262283 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.263245 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.263245 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.263245 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.263245 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.263675 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.263675 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 18 04:57:27.263895 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 18 04:57:27.290821 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80468) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 18 04:57:27.291051 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80468) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 18 04:57:27.291286 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80468) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 18 04:57:27.294132 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80469) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 18 04:57:27.294366 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80469) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 18 04:57:27.294590 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80469) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 18 04:57:27.299841 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80468) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 18 04:57:27.304063 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80469) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 18 04:57:27.328529 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80468) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 18 04:57:27.328759 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80468) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 18 04:57:27.330639 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80469) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 18 04:57:27.330871 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80469) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 18 04:57:27.471660 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80468) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 18 04:57:27.492913 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80469) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 18 04:57:27.522611 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80468) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 18 04:57:27.522844 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80468) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 18 04:57:27.527337 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80469) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 18 04:57:27.527575 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80469) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 18 04:57:27.588359 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80468) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 18 04:57:27.588585 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80468) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 18 04:57:27.588813 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80468) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 18 04:57:27.589033 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80468) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 18 04:57:27.589250 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80468) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 18 04:57:27.589494 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80468) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 18 04:57:27.591396 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80469) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 18 04:57:27.591632 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80469) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 18 04:57:27.591858 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80469) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 18 04:57:27.592082 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80469) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 18 04:57:27.592421 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80469) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 18 04:57:27.593121 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80469) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 18 04:57:27.615928 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80468) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 18 04:57:27.616160 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80468) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 18 04:57:27.616252 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80469) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 18 04:57:27.616388 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80468) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 18 04:57:27.616476 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80469) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 18 04:57:27.616615 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80468) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 18 04:57:27.616698 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80469) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 18 04:57:27.616835 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80468) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 18 04:57:27.616917 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80469) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 18 04:57:27.617053 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80468) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 18 04:57:27.617136 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80469) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 18 04:57:27.617269 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80468) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 18 04:57:27.617352 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80469) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 18 04:57:27.617520 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80469) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 18 04:57:27.628963 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80468) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 18 04:57:27.630070 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80469) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 18 04:57:27.642195 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80468) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 18 04:57:27.642378 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80468) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 18 04:57:27.642599 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80469) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 18 04:57:27.642697 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80468) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 18 04:57:27.642823 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80468) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 18 04:57:27.642905 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80469) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 18 04:57:27.643043 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80469) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 18 04:57:27.643124 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80468) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 18 04:57:27.643258 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80468) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 18 04:57:27.643340 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80469) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 18 04:57:27.643473 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80469) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 18 04:57:27.643561 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80468) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 18 04:57:27.643686 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80469) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 18 04:57:27.643897 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80469) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 18 04:57:27.662700 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80468) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.662907 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80468) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 18 04:57:27.663240 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80468) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.663439 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80468) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 18 04:57:27.663707 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80468) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.663929 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.664016 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80468) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.664139 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 18 04:57:27.664221 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80468) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.664444 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80468) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.664525 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.664656 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 18 04:57:27.664923 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.665179 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.665465 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.665720 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.675511 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80468) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.676459 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80468) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.676736 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80468) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.677015 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80468) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.679050 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.679327 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.679592 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.679867 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.685398 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80468) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 18 04:57:27.688344 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80469) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 18 04:57:27.688676 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80468) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 18 04:57:27.688954 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80468) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.689234 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80468) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.689543 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80468) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.689829 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80468) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.690086 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80468) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.690350 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80468) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.690624 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80468) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.690889 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80468) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.691188 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80468) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.691496 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80468) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.692210 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80469) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 18 04:57:27.692491 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.692849 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.693126 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.693419 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.693679 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.694951 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.695235 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.695508 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.695783 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.696051 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.718372 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80468) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.720001 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80468) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 18 04:57:27.720278 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80468) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.725370 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.726082 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80468) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 18 04:57:27.726473 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80468) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.726756 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80468) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.727024 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80469) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 18 04:57:27.727301 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.732579 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80469) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 18 04:57:27.732961 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.733237 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.734116 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80468) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 18 04:57:27.734398 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80468) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.740033 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80468) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 18 04:57:27.740328 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80468) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.740608 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80468) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.740892 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80468) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.741167 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80468) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.742023 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80468) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.742023 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80468) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.742023 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80468) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.744036 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80469) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 18 04:57:27.744036 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.747924 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80469) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 18 04:57:27.748206 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.748484 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.748764 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.749034 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.749304 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.749594 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.749857 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.761045 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80468) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.761323 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80468) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.766041 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80468) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 18 04:57:27.769719 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80468) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.770118 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80468) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.770931 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.771240 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.775160 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80468) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 18 04:57:27.775803 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80469) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 18 04:57:27.778830 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80468) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 18 04:57:27.780009 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.780411 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.782507 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80468) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.782804 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80468) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.783085 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80468) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.783368 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80468) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.783652 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80468) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.783929 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80468) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.786700 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80469) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 18 04:57:27.790391 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80469) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 18 04:57:27.794177 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.794474 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.794751 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.795027 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.795306 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.795578 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.804054 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80468) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 18 04:57:27.804356 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80468) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.804623 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80468) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.804892 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80468) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.812793 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80468) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.814527 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80468) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.814803 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80468) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.815924 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80469) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 18 04:57:27.816310 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.817072 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.817345 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.818361 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80468) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 18 04:57:27.818643 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80468) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.818921 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80468) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.819197 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80468) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.819477 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80468) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.826492 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.828257 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.828529 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.832325 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80469) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 18 04:57:27.833146 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.833452 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.833596 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80468) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 18 04:57:27.833733 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.833880 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80468) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.834008 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.834167 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80468) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.834447 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80468) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.836140 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80468) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.836429 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80468) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.836723 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80468) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.849345 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80469) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 18 04:57:27.849652 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.849935 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.850211 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.851892 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.852180 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.852474 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.855202 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80468) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 18 04:57:27.858533 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80468) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 18 04:57:27.862085 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80468) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.864371 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80468) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.866031 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80468) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 18 04:57:27.866310 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80468) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.866586 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80468) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.872858 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80469) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 18 04:57:27.874040 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80468) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.875641 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80468) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 18 04:57:27.876144 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80469) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 18 04:57:27.879521 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80468) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.879812 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80468) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.880203 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80468) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.880481 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80468) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.880694 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.882500 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.885066 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80469) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 18 04:57:27.885349 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.885661 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.889644 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80468) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 18 04:57:27.889932 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80468) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.890219 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80468) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.893259 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.895131 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80469) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 18 04:57:27.898160 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80468) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 18 04:57:27.898368 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.898461 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80468) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.898654 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.898761 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80468) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.898951 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80468) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.899033 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.899220 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80468) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.899302 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.899590 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80468) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.899878 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80468) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.900144 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80468) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.900429 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80468) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.900698 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80468) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.900971 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80468) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.901236 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80468) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.901527 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80468) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.901790 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80468) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.909552 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80469) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 18 04:57:27.909867 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.910660 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.918694 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80469) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 18 04:57:27.918968 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.919242 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.919513 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.919780 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.920154 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.920440 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.920720 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.920986 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.921248 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.921535 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.921803 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.922073 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.922387 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 18 04:57:27.940548 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x556e363059c0 pid: 80468 (default app) Jun 18 04:57:27.944828 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86bc122-9931-4a96-9c08-5413607144c3 None None] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:57:27.945126 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86bc122-9931-4a96-9c08-5413607144c3 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:57:27.945422 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86bc122-9931-4a96-9c08-5413607144c3 None None] PATH_INFO: `/v3` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:57:27.950512 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1/1] 10.222.0.23 () {50 vars in 791 bytes} [Sat Jun 18 04:57:27 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:57:27.972353 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x556e363059c0 pid: 80469 (default app) Jun 18 04:57:36.565768 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2423edea-5a68-4b3d-899f-aa50d30bcf86 None None] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:57:36.566323 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2423edea-5a68-4b3d-899f-aa50d30bcf86 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:57:36.566323 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2423edea-5a68-4b3d-899f-aa50d30bcf86 None None] PATH_INFO: `/` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:57:36.567663 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 2/2] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 04:57:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 18 04:57:36.575839 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44ee24f-d11e-4159-af6d-a0aec5e8b85b None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:57:36.576272 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44ee24f-d11e-4159-af6d-a0aec5e8b85b None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:57:36.576412 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44ee24f-d11e-4159-af6d-a0aec5e8b85b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:57:36.645002 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: INFO dbcounter [None req-d44ee24f-d11e-4159-af6d-a0aec5e8b85b None None] Registered counter for database keystone Jun 18 04:57:36.666532 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG oslo_db.sqlalchemy.engines [None req-d44ee24f-d11e-4159-af6d-a0aec5e8b85b None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80469) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 18 04:57:36.672687 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writer thread running {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 18 04:57:36.854597 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG passlib.handlers.bcrypt [None req-d44ee24f-d11e-4159-af6d-a0aec5e8b85b None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80469) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 18 04:57:36.855013 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG passlib.handlers.bcrypt [None req-d44ee24f-d11e-4159-af6d-a0aec5e8b85b None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80469) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 18 04:57:36.910171 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-d44ee24f-d11e-4159-af6d-a0aec5e8b85b None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 04:57:36.911770 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-d44ee24f-d11e-4159-af6d-a0aec5e8b85b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 04:57:37.034911 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1/3] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 04:57:36 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 462 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 18 04:57:37.182282 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2423edea-5a68-4b3d-899f-aa50d30bcf86 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:57:37.253289 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: INFO dbcounter [None req-2423edea-5a68-4b3d-899f-aa50d30bcf86 None None] Registered counter for database keystone Jun 18 04:57:37.274007 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG oslo_db.sqlalchemy.engines [None req-2423edea-5a68-4b3d-899f-aa50d30bcf86 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80468) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 18 04:57:37.280120 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writer thread running {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 18 04:57:37.402337 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2423edea-5a68-4b3d-899f-aa50d30bcf86 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:57:37.406571 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db46bb88-d5a2-4e1e-978f-bbf9c0066cc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:57:37.407223 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db46bb88-d5a2-4e1e-978f-bbf9c0066cc4 admin admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:57:37.407508 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db46bb88-d5a2-4e1e-978f-bbf9c0066cc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:57:37.407786 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db46bb88-d5a2-4e1e-978f-bbf9c0066cc4 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:57:37.416313 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db46bb88-d5a2-4e1e-978f-bbf9c0066cc4 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:57:37.471746 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: WARNING keystone.common.rbac_enforcer.enforcer [None req-db46bb88-d5a2-4e1e-978f-bbf9c0066cc4 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 18 04:57:37.472028 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db46bb88-d5a2-4e1e-978f-bbf9c0066cc4 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:57:37.482179 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 3/4] 10.222.0.23 () {58 vars in 1324 bytes} [Sat Jun 18 04:57:37 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 18 04:57:37.488201 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44ee24f-d11e-4159-af6d-a0aec5e8b85b None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:57:37.503664 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44ee24f-d11e-4159-af6d-a0aec5e8b85b None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:57:37.508633 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842b67ba-24c6-4812-939d-25c44b083e18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:57:37.509769 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842b67ba-24c6-4812-939d-25c44b083e18 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:57:37.510059 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842b67ba-24c6-4812-939d-25c44b083e18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:57:37.510343 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842b67ba-24c6-4812-939d-25c44b083e18 admin admin] PATH_INFO: `/v3/projects/d587d7347f6f4de2882df28dc6b6a574` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:57:37.514360 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842b67ba-24c6-4812-939d-25c44b083e18 admin admin] RBAC: Authorizing `identity:get_project(project_id=d587d7347f6f4de2882df28dc6b6a574)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:57:37.566824 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: WARNING keystone.common.rbac_enforcer.enforcer [None req-842b67ba-24c6-4812-939d-25c44b083e18 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 18 04:57:37.567163 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842b67ba-24c6-4812-939d-25c44b083e18 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:57:37.572676 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 2/5] 10.222.0.23 () {56 vars in 1207 bytes} [Sat Jun 18 04:57:37 2022] GET /identity/v3/projects/d587d7347f6f4de2882df28dc6b6a574 => generated 344 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 18 04:57:39.574928 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77b1ec7-1512-4236-af99-7e3163678fe1 None None] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:57:39.575420 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77b1ec7-1512-4236-af99-7e3163678fe1 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:57:39.575522 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77b1ec7-1512-4236-af99-7e3163678fe1 None None] PATH_INFO: `/` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:57:39.579599 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 4/6] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 04:57:39 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 04:57:39.584863 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752620f2-9b39-4984-bd87-a10f0ed39863 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:57:39.585317 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752620f2-9b39-4984-bd87-a10f0ed39863 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:57:39.585460 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752620f2-9b39-4984-bd87-a10f0ed39863 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:57:39.605246 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-752620f2-9b39-4984-bd87-a10f0ed39863 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 04:57:39.606792 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-752620f2-9b39-4984-bd87-a10f0ed39863 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 04:57:39.615281 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 3/7] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 04:57:39 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 18 04:57:39.637767 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77b1ec7-1512-4236-af99-7e3163678fe1 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:57:39.650147 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77b1ec7-1512-4236-af99-7e3163678fe1 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:57:39.654329 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc0e455-0fdf-4095-a7d8-d6f52dba35a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:57:39.654950 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc0e455-0fdf-4095-a7d8-d6f52dba35a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:57:39.655236 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc0e455-0fdf-4095-a7d8-d6f52dba35a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:57:39.655515 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc0e455-0fdf-4095-a7d8-d6f52dba35a5 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:57:39.663967 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc0e455-0fdf-4095-a7d8-d6f52dba35a5 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:57:39.665794 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc0e455-0fdf-4095-a7d8-d6f52dba35a5 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:57:39.674505 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 5/8] 10.222.0.23 () {58 vars in 1324 bytes} [Sat Jun 18 04:57:39 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 18 04:57:39.680475 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752620f2-9b39-4984-bd87-a10f0ed39863 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:57:39.689639 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752620f2-9b39-4984-bd87-a10f0ed39863 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:57:39.697793 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f855b4cd-5ed4-467b-b034-b49e30ba5799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:57:39.697793 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f855b4cd-5ed4-467b-b034-b49e30ba5799 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:57:39.697793 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f855b4cd-5ed4-467b-b034-b49e30ba5799 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:57:39.697793 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f855b4cd-5ed4-467b-b034-b49e30ba5799 admin admin] PATH_INFO: `/v3/users/5d3e83f3ca7347dea1350fa29bac5d9d` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:57:39.698306 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f855b4cd-5ed4-467b-b034-b49e30ba5799 admin admin] RBAC: Authorizing `identity:get_user(user_id=5d3e83f3ca7347dea1350fa29bac5d9d)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:57:39.699282 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f855b4cd-5ed4-467b-b034-b49e30ba5799 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:57:39.700829 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 4/9] 10.222.0.23 () {56 vars in 1198 bytes} [Sat Jun 18 04:57:39 2022] GET /identity/v3/users/5d3e83f3ca7347dea1350fa29bac5d9d => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:57:41.832946 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4170af5c-e06c-4df6-b79a-f5fce2e60f3e None None] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:57:41.833722 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4170af5c-e06c-4df6-b79a-f5fce2e60f3e None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:57:41.833722 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4170af5c-e06c-4df6-b79a-f5fce2e60f3e None None] PATH_INFO: `/` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:57:41.834999 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 6/10] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 04:57:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 18 04:57:41.839274 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eef9a24-af5f-4cb2-bb15-c648b3ee9d56 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:57:41.839705 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eef9a24-af5f-4cb2-bb15-c648b3ee9d56 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:57:41.839845 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eef9a24-af5f-4cb2-bb15-c648b3ee9d56 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:57:41.839997 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f571cc28-4cce-4adc-9d73-3910d7d1bb29 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:57:41.840287 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f571cc28-4cce-4adc-9d73-3910d7d1bb29 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:57:41.840564 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f571cc28-4cce-4adc-9d73-3910d7d1bb29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:57:41.841197 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 5/11] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 04:57:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 04:57:41.846106 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8f939a-e306-4867-a210-e6b2a9e6f127 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:57:41.846399 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8f939a-e306-4867-a210-e6b2a9e6f127 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:57:41.846723 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8f939a-e306-4867-a210-e6b2a9e6f127 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:57:41.863743 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-ea8f939a-e306-4867-a210-e6b2a9e6f127 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 04:57:41.865762 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-ea8f939a-e306-4867-a210-e6b2a9e6f127 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 04:57:41.881143 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG passlib.handlers.bcrypt [None req-f571cc28-4cce-4adc-9d73-3910d7d1bb29 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80468) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 18 04:57:41.881766 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG passlib.handlers.bcrypt [None req-f571cc28-4cce-4adc-9d73-3910d7d1bb29 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80468) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 18 04:57:41.927518 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-f571cc28-4cce-4adc-9d73-3910d7d1bb29 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 04:57:41.929135 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-f571cc28-4cce-4adc-9d73-3910d7d1bb29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 04:57:41.948503 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 6/12] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 04:57:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 18 04:57:41.972122 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8f939a-e306-4867-a210-e6b2a9e6f127 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:57:41.996945 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:57:42.033161 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 7/13] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 04:57:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 18 04:57:42.057067 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f571cc28-4cce-4adc-9d73-3910d7d1bb29 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:57:42.078566 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8f939a-e306-4867-a210-e6b2a9e6f127 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:57:42.111639 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcb266e-6ebb-4b33-a5c5-09fe7a64e4f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:57:42.112221 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcb266e-6ebb-4b33-a5c5-09fe7a64e4f9 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:57:42.112504 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcb266e-6ebb-4b33-a5c5-09fe7a64e4f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:57:42.112787 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcb266e-6ebb-4b33-a5c5-09fe7a64e4f9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:57:42.158443 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f571cc28-4cce-4adc-9d73-3910d7d1bb29 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:57:42.160686 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:57:42.182889 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcb266e-6ebb-4b33-a5c5-09fe7a64e4f9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:57:42.184391 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcb266e-6ebb-4b33-a5c5-09fe7a64e4f9 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:57:42.195429 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab295b9f-d5c7-440d-8cf0-173fdda3b06c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:57:42.196466 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab295b9f-d5c7-440d-8cf0-173fdda3b06c None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:57:42.196755 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab295b9f-d5c7-440d-8cf0-173fdda3b06c None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:57:42.197036 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab295b9f-d5c7-440d-8cf0-173fdda3b06c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:57:42.207818 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:57:42.211053 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab295b9f-d5c7-440d-8cf0-173fdda3b06c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 18 04:57:42.222176 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application [None req-ab295b9f-d5c7-440d-8cf0-173fdda3b06c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 18 04:57:42.222176 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:57:42.222176 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:57:42.222176 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:57:42.222176 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:57:42.222176 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:57:42.222176 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:57:42.222176 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:57:42.222176 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:57:42.222176 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:57:42.222176 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:57:42.222176 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:57:42.222176 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 18 04:57:42.222176 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 18 04:57:42.222176 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 18 04:57:42.222176 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 18 04:57:42.223240 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 18 04:57:42.223240 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 18 04:57:42.223240 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 18 04:57:42.223240 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 18 04:57:42.223240 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:57:42.223240 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:57:42.223240 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:57:42.223240 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:57:42.223240 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:57:42.223240 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:57:42.223240 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:57:42.223240 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application with Lock( Jun 18 04:57:42.223240 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:57:42.223240 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:57:42.223240 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:57:42.223240 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:57:42.224863 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:57:42.224863 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:57:42.224863 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:57:42.224863 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:57:42.224863 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 18 04:57:42.224863 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 18 04:57:42.224863 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 18 04:57:42.224863 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 18 04:57:42.224863 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 18 04:57:42.224863 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 18 04:57:42.224863 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 18 04:57:42.224863 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 04:57:42.224863 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 8/14] 10.222.0.23 () {56 vars in 1096 bytes} [Sat Jun 18 04:57:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 18 04:57:42.230207 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab295b9f-d5c7-440d-8cf0-173fdda3b06c None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:57:42.288000 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 7/15] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Jun 18 04:57:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 18 04:57:42.293982 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcb266e-6ebb-4b33-a5c5-09fe7a64e4f9 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:57:42.330940 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab295b9f-d5c7-440d-8cf0-173fdda3b06c None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:57:42.366030 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8d9169-94c0-42e3-ad23-d83514e151bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:57:42.366619 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8d9169-94c0-42e3-ad23-d83514e151bf None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:57:42.366928 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8d9169-94c0-42e3-ad23-d83514e151bf None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:57:42.367201 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8d9169-94c0-42e3-ad23-d83514e151bf None admin] PATH_INFO: `/v3/roles` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:57:42.367940 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8d9169-94c0-42e3-ad23-d83514e151bf None admin] RBAC: Adding query filter params (name=admin) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:57:42.368248 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8d9169-94c0-42e3-ad23-d83514e151bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:57:42.369700 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8d9169-94c0-42e3-ad23-d83514e151bf None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:57:42.382540 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 9/16] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Jun 18 04:57:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 18 04:57:42.387914 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8d9169-94c0-42e3-ad23-d83514e151bf None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:57:42.401233 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcb266e-6ebb-4b33-a5c5-09fe7a64e4f9 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:57:42.415102 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:57:42.423636 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:57:42.435000 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbed6a5-ead1-4f49-9b86-785e1546a7d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:57:42.435789 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbed6a5-ead1-4f49-9b86-785e1546a7d7 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:57:42.436078 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbed6a5-ead1-4f49-9b86-785e1546a7d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:57:42.436358 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbed6a5-ead1-4f49-9b86-785e1546a7d7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:57:42.447146 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbed6a5-ead1-4f49-9b86-785e1546a7d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:57:42.448642 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbed6a5-ead1-4f49-9b86-785e1546a7d7 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:57:42.469341 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-ccbed6a5-ead1-4f49-9b86-785e1546a7d7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 04:57:42.469341 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:57:42.469341 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:57:42.469341 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:57:42.469341 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:57:42.469341 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:57:42.469341 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:57:42.469341 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:57:42.469341 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:57:42.469341 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:57:42.469341 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:57:42.469341 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:57:42.469341 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 18 04:57:42.469341 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 18 04:57:42.469341 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 18 04:57:42.470228 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 18 04:57:42.470228 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:57:42.470228 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:57:42.470228 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:57:42.470228 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:57:42.470228 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:57:42.470228 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:57:42.470228 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:57:42.470228 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jun 18 04:57:42.470228 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:57:42.470228 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:57:42.470228 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:57:42.470228 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:57:42.470228 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:57:42.470228 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:57:42.470228 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:57:42.470228 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:57:42.471019 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 18 04:57:42.471019 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 18 04:57:42.471019 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 04:57:42.471019 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 04:57:42.471236 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 8/17] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Jun 18 04:57:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 18 04:57:42.476520 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbed6a5-ead1-4f49-9b86-785e1546a7d7 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:57:42.489674 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8d9169-94c0-42e3-ad23-d83514e151bf None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:57:42.526130 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16378ca4-ac3b-45ed-819b-d1930a0dbe11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:57:42.526992 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16378ca4-ac3b-45ed-819b-d1930a0dbe11 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:57:42.527297 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16378ca4-ac3b-45ed-819b-d1930a0dbe11 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:57:42.527545 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16378ca4-ac3b-45ed-819b-d1930a0dbe11 None admin] PATH_INFO: `/v3/users/5d3e83f3ca7347dea1350fa29bac5d9d` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:57:42.529087 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16378ca4-ac3b-45ed-819b-d1930a0dbe11 None admin] RBAC: Authorizing `identity:get_user(user_id=5d3e83f3ca7347dea1350fa29bac5d9d)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:57:42.530577 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16378ca4-ac3b-45ed-819b-d1930a0dbe11 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:57:42.532194 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 10/18] 10.222.0.23 () {56 vars in 1177 bytes} [Sat Jun 18 04:57:42 2022] GET /identity/v3/users/5d3e83f3ca7347dea1350fa29bac5d9d => generated 247 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 18 04:57:42.538572 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16378ca4-ac3b-45ed-819b-d1930a0dbe11 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:57:42.578615 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbed6a5-ead1-4f49-9b86-785e1546a7d7 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:57:42.615242 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b9c350-1bd0-4841-b9b0-4d3f4ff3f182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:57:42.616249 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b9c350-1bd0-4841-b9b0-4d3f4ff3f182 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:57:42.616249 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b9c350-1bd0-4841-b9b0-4d3f4ff3f182 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:57:42.616445 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b9c350-1bd0-4841-b9b0-4d3f4ff3f182 None admin] PATH_INFO: `/v3/domains` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:57:42.617497 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b9c350-1bd0-4841-b9b0-4d3f4ff3f182 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:57:42.617497 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b9c350-1bd0-4841-b9b0-4d3f4ff3f182 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:57:42.618876 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b9c350-1bd0-4841-b9b0-4d3f4ff3f182 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:57:42.634083 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:57:42.636994 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 9/19] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Jun 18 04:57:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 18 04:57:42.640832 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16378ca4-ac3b-45ed-819b-d1930a0dbe11 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:57:42.679514 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3103aa00-b9a9-40a0-b8c8-0eb553b6b9c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:57:42.680321 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3103aa00-b9a9-40a0-b8c8-0eb553b6b9c7 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:57:42.680608 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3103aa00-b9a9-40a0-b8c8-0eb553b6b9c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:57:42.680887 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3103aa00-b9a9-40a0-b8c8-0eb553b6b9c7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:57:42.682166 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3103aa00-b9a9-40a0-b8c8-0eb553b6b9c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:57:42.683649 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3103aa00-b9a9-40a0-b8c8-0eb553b6b9c7 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:57:42.685358 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 11/20] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Jun 18 04:57:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 18 04:57:42.691398 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b9c350-1bd0-4841-b9b0-4d3f4ff3f182 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:57:42.731779 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:57:42.800446 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b9c350-1bd0-4841-b9b0-4d3f4ff3f182 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:57:42.834544 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55e0a0f-c419-443c-9425-a910d30241e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:57:42.835196 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55e0a0f-c419-443c-9425-a910d30241e1 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:57:42.835483 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55e0a0f-c419-443c-9425-a910d30241e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:57:42.835756 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55e0a0f-c419-443c-9425-a910d30241e1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:57:42.836584 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55e0a0f-c419-443c-9425-a910d30241e1 None admin] RBAC: Adding query filter params (role.id=21c65f2c51bd4dd283d13507d2714467, scope.domain.id=default, user.id=5d3e83f3ca7347dea1350fa29bac5d9d) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:57:42.836898 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55e0a0f-c419-443c-9425-a910d30241e1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:57:42.838399 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55e0a0f-c419-443c-9425-a910d30241e1 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:57:42.845231 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 10/21] 10.222.0.23 () {56 vars in 1322 bytes} [Sat Jun 18 04:57:42 2022] GET /identity/v3/role_assignments?user.id=5d3e83f3ca7347dea1350fa29bac5d9d&scope.domain.id=default&role.id=21c65f2c51bd4dd283d13507d2714467 => generated 233 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:57:45.116891 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fcaaa4-cfe4-4ca0-9480-347d41867cad None None] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:57:45.117448 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fcaaa4-cfe4-4ca0-9480-347d41867cad None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:57:45.117448 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fcaaa4-cfe4-4ca0-9480-347d41867cad None None] PATH_INFO: `/` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:57:45.118823 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 12/22] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 04:57:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 04:57:45.124342 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1792b34-643a-4370-8573-cf90a2faf128 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:57:45.124726 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1792b34-643a-4370-8573-cf90a2faf128 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:57:45.124890 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1792b34-643a-4370-8573-cf90a2faf128 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:57:45.145613 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-b1792b34-643a-4370-8573-cf90a2faf128 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 04:57:45.148104 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-b1792b34-643a-4370-8573-cf90a2faf128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 04:57:45.235598 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 11/23] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 04:57:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 18 04:57:45.274876 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fcaaa4-cfe4-4ca0-9480-347d41867cad None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:57:45.362498 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49b5e2a-e27d-47ae-a241-381272f7d337 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:57:45.362826 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49b5e2a-e27d-47ae-a241-381272f7d337 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:57:45.363065 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49b5e2a-e27d-47ae-a241-381272f7d337 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:57:45.365406 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 12/24] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 04:57:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 18 04:57:45.372351 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcfa0f0-c63f-457c-993a-5670010bc1aa None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:57:45.372583 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcfa0f0-c63f-457c-993a-5670010bc1aa None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:57:45.372872 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcfa0f0-c63f-457c-993a-5670010bc1aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:57:45.408135 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-dbcfa0f0-c63f-457c-993a-5670010bc1aa None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 04:57:45.409341 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-dbcfa0f0-c63f-457c-993a-5670010bc1aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 04:57:45.439675 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fcaaa4-cfe4-4ca0-9480-347d41867cad None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:57:45.458982 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:57:45.502238 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530c884d-f39f-4ac4-87d2-6fc45f3e14c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:57:45.502920 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530c884d-f39f-4ac4-87d2-6fc45f3e14c8 None admin] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:57:45.503209 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530c884d-f39f-4ac4-87d2-6fc45f3e14c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:57:45.503489 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530c884d-f39f-4ac4-87d2-6fc45f3e14c8 None admin] PATH_INFO: `/v3/roles` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:57:45.504267 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530c884d-f39f-4ac4-87d2-6fc45f3e14c8 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:57:45.506223 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530c884d-f39f-4ac4-87d2-6fc45f3e14c8 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:57:45.522955 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 13/25] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 04:57:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 18 04:57:45.524998 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add77d23-c1f7-494a-a6bf-8d76aae90e1e None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:57:45.525610 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add77d23-c1f7-494a-a6bf-8d76aae90e1e None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:57:45.533445 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add77d23-c1f7-494a-a6bf-8d76aae90e1e None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:57:45.533445 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 14/26] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 04:57:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 04:57:45.535083 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5777b2-ca6f-4599-9a1a-11646abe8bdc None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:57:45.535637 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5777b2-ca6f-4599-9a1a-11646abe8bdc None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:57:45.536154 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5777b2-ca6f-4599-9a1a-11646abe8bdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:57:45.556618 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-cd5777b2-ca6f-4599-9a1a-11646abe8bdc None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 04:57:45.558256 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-cd5777b2-ca6f-4599-9a1a-11646abe8bdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 04:57:45.638763 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 15/27] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 04:57:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 18 04:57:45.640129 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5777b2-ca6f-4599-9a1a-11646abe8bdc None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:57:45.735045 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:57:45.746081 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 13/28] 10.222.0.23 () {60 vars in 1131 bytes} [Sat Jun 18 04:57:45 2022] POST /identity/v3/roles => generated 219 bytes in 472 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 04:57:45.749520 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d27be5-cabe-4c7e-8356-ee739c79dc23 None None] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:57:45.749520 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d27be5-cabe-4c7e-8356-ee739c79dc23 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:57:45.749520 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d27be5-cabe-4c7e-8356-ee739c79dc23 None None] PATH_INFO: `/` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:57:45.750051 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 14/29] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 04:57:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 04:57:45.750943 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d27be5-cabe-4c7e-8356-ee739c79dc23 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:57:45.756255 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5777b2-ca6f-4599-9a1a-11646abe8bdc None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:57:45.787525 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:57:45.816584 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8f5347-f357-4625-a694-8022d600e219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:57:45.819707 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8f5347-f357-4625-a694-8022d600e219 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:57:45.821101 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8f5347-f357-4625-a694-8022d600e219 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:57:45.821575 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8f5347-f357-4625-a694-8022d600e219 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:57:45.850408 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8f5347-f357-4625-a694-8022d600e219 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 18 04:57:45.861119 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-0f8f5347-f357-4625-a694-8022d600e219 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 18 04:57:45.861119 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:57:45.861119 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:57:45.861119 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:57:45.861119 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:57:45.861119 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:57:45.861119 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:57:45.861119 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:57:45.861119 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:57:45.861119 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:57:45.861119 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:57:45.861119 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:57:45.861119 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 18 04:57:45.861119 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 18 04:57:45.861119 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 18 04:57:45.861119 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 18 04:57:45.862134 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 18 04:57:45.862134 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 18 04:57:45.862134 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 18 04:57:45.862134 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 18 04:57:45.862134 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:57:45.862134 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:57:45.862134 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:57:45.862134 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:57:45.862134 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:57:45.862134 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:57:45.862134 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:57:45.862134 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jun 18 04:57:45.862134 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:57:45.862134 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:57:45.862134 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:57:45.862134 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:57:45.862910 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:57:45.862910 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:57:45.862910 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:57:45.862910 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:57:45.862910 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 18 04:57:45.862910 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 18 04:57:45.862910 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 18 04:57:45.862910 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 18 04:57:45.862910 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 18 04:57:45.862910 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 18 04:57:45.862910 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 18 04:57:45.862910 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 04:57:45.862910 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 16/30] 10.222.0.23 () {56 vars in 1096 bytes} [Sat Jun 18 04:57:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:57:45.865023 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d8a6c9-e8b1-4f73-b480-befbf12dacf2 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:57:45.865468 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d8a6c9-e8b1-4f73-b480-befbf12dacf2 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:57:45.865819 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d8a6c9-e8b1-4f73-b480-befbf12dacf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:57:45.902657 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-f6d8a6c9-e8b1-4f73-b480-befbf12dacf2 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 04:57:45.906836 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-f6d8a6c9-e8b1-4f73-b480-befbf12dacf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 04:57:45.964671 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d27be5-cabe-4c7e-8356-ee739c79dc23 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:57:46.003205 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3cf11f-3cc5-44b5-9901-14edf62368b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:57:46.003874 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3cf11f-3cc5-44b5-9901-14edf62368b0 None admin] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:57:46.004166 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3cf11f-3cc5-44b5-9901-14edf62368b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:57:46.004449 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3cf11f-3cc5-44b5-9901-14edf62368b0 None admin] PATH_INFO: `/v3/roles` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:57:46.005274 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3cf11f-3cc5-44b5-9901-14edf62368b0 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:57:46.007169 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3cf11f-3cc5-44b5-9901-14edf62368b0 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:57:46.015941 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 17/31] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 04:57:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 18 04:57:46.017244 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d8a6c9-e8b1-4f73-b480-befbf12dacf2 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:57:46.092595 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 15/32] 10.222.0.23 () {60 vars in 1131 bytes} [Sat Jun 18 04:57:45 2022] POST /identity/v3/roles => generated 225 bytes in 342 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 04:57:46.097098 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3cf11f-3cc5-44b5-9901-14edf62368b0 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:57:46.111562 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:57:46.137164 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d8a6c9-e8b1-4f73-b480-befbf12dacf2 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:57:46.177830 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08dfb07-6879-4b7e-892c-4f4ff2ed2158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:57:46.178770 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08dfb07-6879-4b7e-892c-4f4ff2ed2158 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:57:46.179058 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08dfb07-6879-4b7e-892c-4f4ff2ed2158 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:57:46.179387 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08dfb07-6879-4b7e-892c-4f4ff2ed2158 None admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:57:46.180224 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08dfb07-6879-4b7e-892c-4f4ff2ed2158 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:57:46.180543 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08dfb07-6879-4b7e-892c-4f4ff2ed2158 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:57:46.182021 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08dfb07-6879-4b7e-892c-4f4ff2ed2158 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:57:46.197255 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 18/33] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Jun 18 04:57:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:57:46.207150 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08dfb07-6879-4b7e-892c-4f4ff2ed2158 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:57:46.225368 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:57:46.266012 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3cf11f-3cc5-44b5-9901-14edf62368b0 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:57:46.310903 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea040c8b-9e29-43a7-866b-d1398ab09374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:57:46.311722 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea040c8b-9e29-43a7-866b-d1398ab09374 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:57:46.312015 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea040c8b-9e29-43a7-866b-d1398ab09374 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:57:46.312300 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea040c8b-9e29-43a7-866b-d1398ab09374 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:57:46.335538 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea040c8b-9e29-43a7-866b-d1398ab09374 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:57:46.337065 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea040c8b-9e29-43a7-866b-d1398ab09374 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:57:46.352140 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08dfb07-6879-4b7e-892c-4f4ff2ed2158 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:57:46.357232 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application [None req-ea040c8b-9e29-43a7-866b-d1398ab09374 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 04:57:46.357232 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:57:46.357232 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:57:46.357232 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:57:46.357232 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:57:46.357232 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:57:46.357232 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:57:46.357232 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:57:46.357232 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:57:46.357232 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:57:46.357232 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:57:46.357232 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:57:46.357232 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 18 04:57:46.357232 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 18 04:57:46.357232 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 18 04:57:46.358061 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 18 04:57:46.358061 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:57:46.358061 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:57:46.358061 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:57:46.358061 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:57:46.358061 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:57:46.358061 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:57:46.358061 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:57:46.358061 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application with Lock( Jun 18 04:57:46.358061 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:57:46.358061 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:57:46.358061 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:57:46.358061 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:57:46.358061 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:57:46.358061 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:57:46.358061 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:57:46.358061 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:57:46.358845 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 18 04:57:46.358845 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 18 04:57:46.358845 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 04:57:46.358845 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 04:57:46.358845 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 16/34] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Jun 18 04:57:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:57:46.365824 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea040c8b-9e29-43a7-866b-d1398ab09374 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:57:46.391937 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed16df9-a5c6-4eae-90e1-57ed346309fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:57:46.392836 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed16df9-a5c6-4eae-90e1-57ed346309fb None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:57:46.393126 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed16df9-a5c6-4eae-90e1-57ed346309fb None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:57:46.393428 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed16df9-a5c6-4eae-90e1-57ed346309fb None admin] PATH_INFO: `/v3/users/5d3e83f3ca7347dea1350fa29bac5d9d` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:57:46.394778 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed16df9-a5c6-4eae-90e1-57ed346309fb None admin] RBAC: Authorizing `identity:get_user(user_id=5d3e83f3ca7347dea1350fa29bac5d9d)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:57:46.396866 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed16df9-a5c6-4eae-90e1-57ed346309fb None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:57:46.398827 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 19/35] 10.222.0.23 () {56 vars in 1177 bytes} [Sat Jun 18 04:57:46 2022] GET /identity/v3/users/5d3e83f3ca7347dea1350fa29bac5d9d => generated 247 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:57:46.407823 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed16df9-a5c6-4eae-90e1-57ed346309fb None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:57:46.421087 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:57:46.497852 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea040c8b-9e29-43a7-866b-d1398ab09374 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:57:46.522473 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed16df9-a5c6-4eae-90e1-57ed346309fb None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:57:46.539121 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4094074f-c5a8-4367-900b-8fa4ceeacfb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:57:46.539805 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4094074f-c5a8-4367-900b-8fa4ceeacfb5 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:57:46.540099 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4094074f-c5a8-4367-900b-8fa4ceeacfb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:57:46.540485 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4094074f-c5a8-4367-900b-8fa4ceeacfb5 None admin] PATH_INFO: `/v3/domains` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:57:46.541336 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4094074f-c5a8-4367-900b-8fa4ceeacfb5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:57:46.541687 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4094074f-c5a8-4367-900b-8fa4ceeacfb5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:57:46.543097 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4094074f-c5a8-4367-900b-8fa4ceeacfb5 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:57:46.545102 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:57:46.560716 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebff21a-7f75-4ee1-a200-30c18178456a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:57:46.561561 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebff21a-7f75-4ee1-a200-30c18178456a None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:57:46.562023 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebff21a-7f75-4ee1-a200-30c18178456a None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:57:46.562200 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebff21a-7f75-4ee1-a200-30c18178456a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:57:46.563406 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebff21a-7f75-4ee1-a200-30c18178456a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:57:46.563741 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 17/36] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Jun 18 04:57:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:57:46.565081 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebff21a-7f75-4ee1-a200-30c18178456a None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:57:46.566833 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 20/37] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Jun 18 04:57:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 18 04:57:46.572206 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4094074f-c5a8-4367-900b-8fa4ceeacfb5 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:57:46.574325 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebff21a-7f75-4ee1-a200-30c18178456a None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:57:46.662572 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:57:46.683113 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4094074f-c5a8-4367-900b-8fa4ceeacfb5 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:57:46.687960 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebff21a-7f75-4ee1-a200-30c18178456a None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:57:46.719580 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7749d0c1-da6b-4f7b-a8df-73ce707b84b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:57:46.720242 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7749d0c1-da6b-4f7b-a8df-73ce707b84b9 None admin] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:57:46.720531 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7749d0c1-da6b-4f7b-a8df-73ce707b84b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:57:46.722286 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7749d0c1-da6b-4f7b-a8df-73ce707b84b9 None admin] PATH_INFO: `/v3/projects` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:57:46.722286 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7749d0c1-da6b-4f7b-a8df-73ce707b84b9 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:57:46.722721 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bd5935-c8a7-4940-8d0c-1a6fdd776443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:57:46.723187 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7749d0c1-da6b-4f7b-a8df-73ce707b84b9 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:57:46.723591 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bd5935-c8a7-4940-8d0c-1a6fdd776443 None admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:57:46.724421 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bd5935-c8a7-4940-8d0c-1a6fdd776443 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:57:46.724712 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bd5935-c8a7-4940-8d0c-1a6fdd776443 None admin] PATH_INFO: `/v3/domains/default/users/5d3e83f3ca7347dea1350fa29bac5d9d/roles/21c65f2c51bd4dd283d13507d2714467` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:57:46.726872 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bd5935-c8a7-4940-8d0c-1a6fdd776443 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=5d3e83f3ca7347dea1350fa29bac5d9d, role_id=21c65f2c51bd4dd283d13507d2714467)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:57:46.728318 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bd5935-c8a7-4940-8d0c-1a6fdd776443 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:57:46.842889 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 18/38] 10.222.0.23 () {60 vars in 1141 bytes} [Sat Jun 18 04:57:46 2022] POST /identity/v3/projects => generated 301 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 04:57:46.844243 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23034ba2-1f55-41ca-9d8d-4211b4b9e0be None None] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:57:46.844549 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23034ba2-1f55-41ca-9d8d-4211b4b9e0be None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:57:46.844827 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23034ba2-1f55-41ca-9d8d-4211b4b9e0be None None] PATH_INFO: `/` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:57:46.846113 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 19/39] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 04:57:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 04:57:46.847243 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff0ff76-6c6a-4436-9dcb-c6f38b6fdef1 None None] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:57:46.847596 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff0ff76-6c6a-4436-9dcb-c6f38b6fdef1 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:57:46.847922 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff0ff76-6c6a-4436-9dcb-c6f38b6fdef1 None None] PATH_INFO: `/` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:57:46.853481 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 20/40] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 04:57:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 04:57:46.853481 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dbb678-9c97-4f78-9f18-ae8c61f5f95c None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:57:46.853481 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dbb678-9c97-4f78-9f18-ae8c61f5f95c None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:57:46.853481 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dbb678-9c97-4f78-9f18-ae8c61f5f95c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:57:46.872074 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-56dbb678-9c97-4f78-9f18-ae8c61f5f95c None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 04:57:46.873677 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-56dbb678-9c97-4f78-9f18-ae8c61f5f95c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 04:57:46.921219 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:57:46.934572 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 21/41] 10.222.0.23 () {58 vars in 1361 bytes} [Sat Jun 18 04:57:46 2022] PUT /identity/v3/domains/default/users/5d3e83f3ca7347dea1350fa29bac5d9d/roles/21c65f2c51bd4dd283d13507d2714467 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 18 04:57:46.940043 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead0a35e-e331-4b12-9326-696a34cb1d09 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:57:46.940043 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead0a35e-e331-4b12-9326-696a34cb1d09 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:57:46.940043 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead0a35e-e331-4b12-9326-696a34cb1d09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:57:46.961711 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 21/42] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 04:57:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 18 04:57:46.988361 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dbb678-9c97-4f78-9f18-ae8c61f5f95c None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:57:47.080733 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-ead0a35e-e331-4b12-9326-696a34cb1d09 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 04:57:47.082351 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-ead0a35e-e331-4b12-9326-696a34cb1d09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 04:57:47.121186 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dbb678-9c97-4f78-9f18-ae8c61f5f95c None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:57:47.159044 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b674b82e-9105-41f8-86c0-72e8132f4912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:57:47.160007 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b674b82e-9105-41f8-86c0-72e8132f4912 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:57:47.160397 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b674b82e-9105-41f8-86c0-72e8132f4912 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:57:47.160777 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b674b82e-9105-41f8-86c0-72e8132f4912 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:57:47.162048 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 22/43] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 04:57:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 18 04:57:47.162496 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b674b82e-9105-41f8-86c0-72e8132f4912 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:57:47.164080 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b674b82e-9105-41f8-86c0-72e8132f4912 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:57:47.169424 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 22/44] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Jun 18 04:57:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:57:47.171478 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead0a35e-e331-4b12-9326-696a34cb1d09 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:57:47.184376 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b674b82e-9105-41f8-86c0-72e8132f4912 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:57:47.237010 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:57:47.244879 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:57:47.281862 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead0a35e-e331-4b12-9326-696a34cb1d09 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:57:47.299786 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b674b82e-9105-41f8-86c0-72e8132f4912 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:57:47.323231 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b603fcc3-263d-4c54-afb9-fcd968dff196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:57:47.324704 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b603fcc3-263d-4c54-afb9-fcd968dff196 None admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:57:47.325338 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b603fcc3-263d-4c54-afb9-fcd968dff196 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:57:47.325889 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b603fcc3-263d-4c54-afb9-fcd968dff196 None admin] PATH_INFO: `/v3/projects` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:57:47.326925 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b603fcc3-263d-4c54-afb9-fcd968dff196 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:57:47.329336 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b603fcc3-263d-4c54-afb9-fcd968dff196 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:57:47.336015 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd91e47-b905-416d-b4db-c06aa92539fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:57:47.336554 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd91e47-b905-416d-b4db-c06aa92539fb None admin] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:57:47.336841 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd91e47-b905-416d-b4db-c06aa92539fb None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:57:47.337121 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd91e47-b905-416d-b4db-c06aa92539fb None admin] PATH_INFO: `/v3/roles` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:57:47.337936 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd91e47-b905-416d-b4db-c06aa92539fb None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:57:47.339314 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd91e47-b905-416d-b4db-c06aa92539fb None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:57:47.448598 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 23/45] 10.222.0.23 () {60 vars in 1131 bytes} [Sat Jun 18 04:57:47 2022] POST /identity/v3/roles => generated 223 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 18 04:57:47.482662 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 23/46] 10.222.0.23 () {60 vars in 1141 bytes} [Sat Jun 18 04:57:47 2022] POST /identity/v3/projects => generated 312 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 18 04:57:49.033368 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f937161-4b3d-4277-95d8-d0f92e8499b4 None None] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:57:49.033902 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f937161-4b3d-4277-95d8-d0f92e8499b4 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:57:49.033985 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f937161-4b3d-4277-95d8-d0f92e8499b4 None None] PATH_INFO: `/` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:57:49.035367 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 24/47] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 04:57:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 04:57:49.040974 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332b48c1-9368-4772-83e2-c719c40e0a15 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:57:49.041428 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332b48c1-9368-4772-83e2-c719c40e0a15 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:57:49.041578 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332b48c1-9368-4772-83e2-c719c40e0a15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:57:49.059008 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-332b48c1-9368-4772-83e2-c719c40e0a15 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 04:57:49.060190 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-332b48c1-9368-4772-83e2-c719c40e0a15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 04:57:49.126359 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 24/48] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 04:57:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 18 04:57:49.149989 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f937161-4b3d-4277-95d8-d0f92e8499b4 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:57:49.253453 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f937161-4b3d-4277-95d8-d0f92e8499b4 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:57:49.256446 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:57:49.287454 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ec6dc3-e45e-4afd-bcec-77e52405e5a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:57:49.288168 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ec6dc3-e45e-4afd-bcec-77e52405e5a1 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:57:49.288456 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ec6dc3-e45e-4afd-bcec-77e52405e5a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:57:49.288737 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ec6dc3-e45e-4afd-bcec-77e52405e5a1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:57:49.298592 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ec6dc3-e45e-4afd-bcec-77e52405e5a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 18 04:57:49.306215 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application [None req-10ec6dc3-e45e-4afd-bcec-77e52405e5a1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 18 04:57:49.306215 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:57:49.306215 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:57:49.306215 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:57:49.306215 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:57:49.306215 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:57:49.306215 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:57:49.306215 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:57:49.306215 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:57:49.306215 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:57:49.306215 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:57:49.306215 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:57:49.306215 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 18 04:57:49.306215 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 18 04:57:49.306215 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 18 04:57:49.306215 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 18 04:57:49.307053 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 18 04:57:49.307053 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 18 04:57:49.307053 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 18 04:57:49.307053 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 18 04:57:49.307053 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:57:49.307053 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:57:49.307053 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:57:49.307053 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:57:49.307053 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:57:49.307053 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:57:49.307053 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:57:49.307053 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application with Lock( Jun 18 04:57:49.307053 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:57:49.307053 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:57:49.307053 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:57:49.307053 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:57:49.308003 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:57:49.308003 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:57:49.308003 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:57:49.308003 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:57:49.308003 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 18 04:57:49.308003 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 18 04:57:49.308003 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 18 04:57:49.308003 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 18 04:57:49.308003 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 18 04:57:49.308003 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 18 04:57:49.308003 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 18 04:57:49.308003 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 04:57:49.308003 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 25/49] 10.222.0.23 () {56 vars in 1096 bytes} [Sat Jun 18 04:57:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:57:49.313120 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332b48c1-9368-4772-83e2-c719c40e0a15 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:57:49.362068 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:57:49.415287 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332b48c1-9368-4772-83e2-c719c40e0a15 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:57:49.448978 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbd6f7b-6ab5-46be-92d6-41df86929fc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:57:49.449668 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbd6f7b-6ab5-46be-92d6-41df86929fc3 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:57:49.449958 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbd6f7b-6ab5-46be-92d6-41df86929fc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:57:49.450239 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbd6f7b-6ab5-46be-92d6-41df86929fc3 None admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:57:49.450998 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbd6f7b-6ab5-46be-92d6-41df86929fc3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:57:49.451313 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbd6f7b-6ab5-46be-92d6-41df86929fc3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:57:49.452718 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbd6f7b-6ab5-46be-92d6-41df86929fc3 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:57:49.460845 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 25/50] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Jun 18 04:57:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:57:49.466319 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ec6dc3-e45e-4afd-bcec-77e52405e5a1 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:57:49.538129 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb65610-65b0-47a1-94e5-7d32d2b16dce None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:57:49.538347 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb65610-65b0-47a1-94e5-7d32d2b16dce None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:57:49.538636 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb65610-65b0-47a1-94e5-7d32d2b16dce None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:57:49.540013 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 26/51] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 04:57:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 18 04:57:49.544727 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c82c10-9d58-48b7-b7a8-a96c27ebe4bb None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:57:49.545021 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c82c10-9d58-48b7-b7a8-a96c27ebe4bb None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:57:49.545303 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c82c10-9d58-48b7-b7a8-a96c27ebe4bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:57:49.562426 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-36c82c10-9d58-48b7-b7a8-a96c27ebe4bb None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 04:57:49.563630 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-36c82c10-9d58-48b7-b7a8-a96c27ebe4bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 04:57:49.569462 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ec6dc3-e45e-4afd-bcec-77e52405e5a1 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:57:49.602903 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094ee8f5-c504-4a11-953e-752acdcb77a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:57:49.603764 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094ee8f5-c504-4a11-953e-752acdcb77a0 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:57:49.604052 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094ee8f5-c504-4a11-953e-752acdcb77a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:57:49.604329 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094ee8f5-c504-4a11-953e-752acdcb77a0 None admin] PATH_INFO: `/v3/users/5d3e83f3ca7347dea1350fa29bac5d9d` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:57:49.605563 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094ee8f5-c504-4a11-953e-752acdcb77a0 None admin] RBAC: Authorizing `identity:get_user(user_id=5d3e83f3ca7347dea1350fa29bac5d9d)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:57:49.607058 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094ee8f5-c504-4a11-953e-752acdcb77a0 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:57:49.608678 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 26/52] 10.222.0.23 () {56 vars in 1177 bytes} [Sat Jun 18 04:57:49 2022] GET /identity/v3/users/5d3e83f3ca7347dea1350fa29bac5d9d => generated 247 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:57:49.614205 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094ee8f5-c504-4a11-953e-752acdcb77a0 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:57:49.633209 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 27/53] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 04:57:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 18 04:57:49.646310 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:57:49.656925 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c82c10-9d58-48b7-b7a8-a96c27ebe4bb None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:57:49.700635 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:57:49.762882 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c82c10-9d58-48b7-b7a8-a96c27ebe4bb None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:57:49.799879 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b94b52-6619-4847-b7b8-241f129f0a6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:57:49.800653 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b94b52-6619-4847-b7b8-241f129f0a6a None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:57:49.800954 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b94b52-6619-4847-b7b8-241f129f0a6a None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:57:49.801227 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b94b52-6619-4847-b7b8-241f129f0a6a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:57:49.802978 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b94b52-6619-4847-b7b8-241f129f0a6a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:57:49.804448 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b94b52-6619-4847-b7b8-241f129f0a6a None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:57:49.806123 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 28/54] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Jun 18 04:57:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 18 04:57:49.811660 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b94b52-6619-4847-b7b8-241f129f0a6a None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:57:49.820962 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094ee8f5-c504-4a11-953e-752acdcb77a0 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:57:49.854969 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7109e3c-e69a-426a-b56e-c18d507fbef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:57:49.855760 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7109e3c-e69a-426a-b56e-c18d507fbef2 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:57:49.856058 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7109e3c-e69a-426a-b56e-c18d507fbef2 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:57:49.856340 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7109e3c-e69a-426a-b56e-c18d507fbef2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:57:49.857575 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7109e3c-e69a-426a-b56e-c18d507fbef2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:57:49.859124 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7109e3c-e69a-426a-b56e-c18d507fbef2 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:57:49.860752 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 27/55] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Jun 18 04:57:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 18 04:57:49.867293 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7109e3c-e69a-426a-b56e-c18d507fbef2 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:57:49.916029 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b94b52-6619-4847-b7b8-241f129f0a6a None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:57:49.925669 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:57:49.950499 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b984dfd-8573-41c1-a054-8825a9442343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:57:49.951162 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b984dfd-8573-41c1-a054-8825a9442343 None admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:57:49.951450 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b984dfd-8573-41c1-a054-8825a9442343 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:57:49.951735 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b984dfd-8573-41c1-a054-8825a9442343 None admin] PATH_INFO: `/v3/projects` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:57:49.952447 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b984dfd-8573-41c1-a054-8825a9442343 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:57:49.953954 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b984dfd-8573-41c1-a054-8825a9442343 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:57:49.963815 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:57:49.969472 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7109e3c-e69a-426a-b56e-c18d507fbef2 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:57:50.001443 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26aa08b1-594f-478a-999d-d00539038500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:57:50.002118 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aa08b1-594f-478a-999d-d00539038500 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:57:50.002411 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aa08b1-594f-478a-999d-d00539038500 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:57:50.002694 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aa08b1-594f-478a-999d-d00539038500 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:57:50.003525 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26aa08b1-594f-478a-999d-d00539038500 None admin] RBAC: Adding query filter params (role.id=21c65f2c51bd4dd283d13507d2714467, scope.domain.id=default, user.id=5d3e83f3ca7347dea1350fa29bac5d9d) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:57:50.003844 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26aa08b1-594f-478a-999d-d00539038500 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:57:50.005313 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26aa08b1-594f-478a-999d-d00539038500 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:57:50.014938 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 28/56] 10.222.0.23 () {56 vars in 1322 bytes} [Sat Jun 18 04:57:49 2022] GET /identity/v3/role_assignments?user.id=5d3e83f3ca7347dea1350fa29bac5d9d&scope.domain.id=default&role.id=21c65f2c51bd4dd283d13507d2714467 => generated 530 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 18 04:57:50.066951 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 29/57] 10.222.0.23 () {60 vars in 1140 bytes} [Sat Jun 18 04:57:49 2022] POST /identity/v3/projects => generated 298 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 18 04:57:52.118573 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8647323c-97ff-4f5d-a21f-49f4208da55f None None] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:57:52.119081 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8647323c-97ff-4f5d-a21f-49f4208da55f None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:57:52.119182 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8647323c-97ff-4f5d-a21f-49f4208da55f None None] PATH_INFO: `/` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:57:52.120512 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 29/58] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 04:57:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 04:57:52.125621 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028cc15e-ba76-4aa8-b2ad-7c97d6004bc8 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:57:52.126024 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028cc15e-ba76-4aa8-b2ad-7c97d6004bc8 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:57:52.126187 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028cc15e-ba76-4aa8-b2ad-7c97d6004bc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:57:52.144204 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-028cc15e-ba76-4aa8-b2ad-7c97d6004bc8 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 04:57:52.145446 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-028cc15e-ba76-4aa8-b2ad-7c97d6004bc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 04:57:52.209210 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 30/59] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 04:57:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 18 04:57:52.234369 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8647323c-97ff-4f5d-a21f-49f4208da55f None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:57:52.337235 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8647323c-97ff-4f5d-a21f-49f4208da55f None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:57:52.370549 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219278b7-6e83-46c4-9ca5-6387b40ed90e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:57:52.371260 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219278b7-6e83-46c4-9ca5-6387b40ed90e None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:57:52.371544 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219278b7-6e83-46c4-9ca5-6387b40ed90e None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:57:52.371820 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219278b7-6e83-46c4-9ca5-6387b40ed90e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:57:52.372993 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219278b7-6e83-46c4-9ca5-6387b40ed90e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:57:52.374478 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219278b7-6e83-46c4-9ca5-6387b40ed90e None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:57:52.376087 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 30/60] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Jun 18 04:57:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:57:52.382144 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028cc15e-ba76-4aa8-b2ad-7c97d6004bc8 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:57:52.469349 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:57:52.478576 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028cc15e-ba76-4aa8-b2ad-7c97d6004bc8 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:57:52.511874 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1e087a-c26b-4c14-8e01-102e0446d9d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:57:52.512540 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1e087a-c26b-4c14-8e01-102e0446d9d9 None admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:57:52.512825 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1e087a-c26b-4c14-8e01-102e0446d9d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:57:52.513106 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1e087a-c26b-4c14-8e01-102e0446d9d9 None admin] PATH_INFO: `/v3/users` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:57:52.513856 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1e087a-c26b-4c14-8e01-102e0446d9d9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:57:52.515364 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1e087a-c26b-4c14-8e01-102e0446d9d9 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:57:52.617505 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 31/61] 10.222.0.23 () {60 vars in 1132 bytes} [Sat Jun 18 04:57:52 2022] POST /identity/v3/users => generated 275 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 04:57:55.619984 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab1eaea-95bb-4e81-a892-00e9981a1d74 None None] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:57:55.619984 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab1eaea-95bb-4e81-a892-00e9981a1d74 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:57:55.619984 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab1eaea-95bb-4e81-a892-00e9981a1d74 None None] PATH_INFO: `/` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:57:55.619984 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 31/62] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 04:57:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 04:57:55.625496 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ef54ba-1c61-47cd-a722-24a49836a168 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:57:55.625873 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ef54ba-1c61-47cd-a722-24a49836a168 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:57:55.626071 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ef54ba-1c61-47cd-a722-24a49836a168 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:57:55.650431 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-a5ef54ba-1c61-47cd-a722-24a49836a168 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 04:57:55.651845 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-a5ef54ba-1c61-47cd-a722-24a49836a168 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 04:57:55.754365 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 32/63] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 04:57:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 18 04:57:55.796438 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab1eaea-95bb-4e81-a892-00e9981a1d74 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:57:55.844528 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:57:55.946078 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab1eaea-95bb-4e81-a892-00e9981a1d74 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:57:55.991688 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4cec14-2735-4e53-aa67-e07ad6718ed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:57:55.992495 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4cec14-2735-4e53-aa67-e07ad6718ed1 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:57:55.992785 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4cec14-2735-4e53-aa67-e07ad6718ed1 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:57:55.993064 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4cec14-2735-4e53-aa67-e07ad6718ed1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:57:56.009323 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4cec14-2735-4e53-aa67-e07ad6718ed1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 18 04:57:56.017051 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application [None req-1c4cec14-2735-4e53-aa67-e07ad6718ed1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 18 04:57:56.017051 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:57:56.017051 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:57:56.017051 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:57:56.017051 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:57:56.017051 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:57:56.017051 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:57:56.017051 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:57:56.017051 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:57:56.017051 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:57:56.017051 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:57:56.017051 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:57:56.017051 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 18 04:57:56.017051 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 18 04:57:56.017051 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 18 04:57:56.017051 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 18 04:57:56.017965 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 18 04:57:56.017965 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 18 04:57:56.017965 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 18 04:57:56.017965 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 18 04:57:56.017965 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:57:56.017965 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:57:56.017965 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:57:56.017965 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:57:56.017965 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:57:56.017965 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:57:56.017965 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:57:56.017965 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application with Lock( Jun 18 04:57:56.017965 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:57:56.017965 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:57:56.017965 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:57:56.017965 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:57:56.018718 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:57:56.018718 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:57:56.018718 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:57:56.018718 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:57:56.018718 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 18 04:57:56.018718 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 18 04:57:56.018718 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 18 04:57:56.018718 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 18 04:57:56.018718 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 18 04:57:56.018718 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 18 04:57:56.018718 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 18 04:57:56.018718 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 04:57:56.018718 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 32/64] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Jun 18 04:57:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:57:56.025452 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ef54ba-1c61-47cd-a722-24a49836a168 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:57:56.066199 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd25ce0d-5e19-4c22-988f-fd18aee5c9c7 None None] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:57:56.066487 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd25ce0d-5e19-4c22-988f-fd18aee5c9c7 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:57:56.066995 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd25ce0d-5e19-4c22-988f-fd18aee5c9c7 None None] PATH_INFO: `/` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:57:56.068565 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 33/65] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 04:57:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 04:57:56.077484 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372443c2-683b-4f40-96f5-2b74fac87978 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:57:56.077992 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372443c2-683b-4f40-96f5-2b74fac87978 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:57:56.078448 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372443c2-683b-4f40-96f5-2b74fac87978 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:57:56.106381 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-372443c2-683b-4f40-96f5-2b74fac87978 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 04:57:56.107867 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-372443c2-683b-4f40-96f5-2b74fac87978 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 04:57:56.194262 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:57:56.232599 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ef54ba-1c61-47cd-a722-24a49836a168 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:57:56.238886 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 34/66] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 04:57:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 18 04:57:56.241172 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753c1f94-c0dc-40cb-8633-7bfbd17559dc None None] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:57:56.241716 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753c1f94-c0dc-40cb-8633-7bfbd17559dc None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:57:56.242388 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753c1f94-c0dc-40cb-8633-7bfbd17559dc None None] PATH_INFO: `/` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:57:56.243985 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 35/67] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 04:57:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 04:57:56.251638 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346785ab-bae2-4549-b76e-1571112d6c26 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:57:56.252169 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346785ab-bae2-4549-b76e-1571112d6c26 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:57:56.252666 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346785ab-bae2-4549-b76e-1571112d6c26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:57:56.275319 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1347c010-e46e-4bba-8b8e-dd514f9735b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:57:56.275999 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1347c010-e46e-4bba-8b8e-dd514f9735b0 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:57:56.276289 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1347c010-e46e-4bba-8b8e-dd514f9735b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:57:56.276572 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1347c010-e46e-4bba-8b8e-dd514f9735b0 None admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:57:56.277335 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1347c010-e46e-4bba-8b8e-dd514f9735b0 None admin] RBAC: Adding query filter params (name=member) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:57:56.277685 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1347c010-e46e-4bba-8b8e-dd514f9735b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:57:56.279102 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1347c010-e46e-4bba-8b8e-dd514f9735b0 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:57:56.282494 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-346785ab-bae2-4549-b76e-1571112d6c26 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 04:57:56.283927 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-346785ab-bae2-4549-b76e-1571112d6c26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 04:57:56.288341 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 33/68] 10.222.0.23 () {56 vars in 1101 bytes} [Sat Jun 18 04:57:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:57:56.289366 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1347c010-e46e-4bba-8b8e-dd514f9735b0 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:57:56.303972 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:57:56.359412 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 36/69] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 04:57:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 18 04:57:56.361104 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346785ab-bae2-4549-b76e-1571112d6c26 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:57:56.407167 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1347c010-e46e-4bba-8b8e-dd514f9735b0 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:57:56.441340 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e775e7c8-f4c8-456b-a5e4-a66e6ab7df53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:57:56.442153 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e775e7c8-f4c8-456b-a5e4-a66e6ab7df53 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:57:56.442441 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e775e7c8-f4c8-456b-a5e4-a66e6ab7df53 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:57:56.442716 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e775e7c8-f4c8-456b-a5e4-a66e6ab7df53 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:57:56.450495 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:57:56.454873 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e775e7c8-f4c8-456b-a5e4-a66e6ab7df53 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 18 04:57:56.462806 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-e775e7c8-f4c8-456b-a5e4-a66e6ab7df53 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 18 04:57:56.462806 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:57:56.462806 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:57:56.462806 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:57:56.462806 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:57:56.462806 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:57:56.462806 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:57:56.462806 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:57:56.462806 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:57:56.462806 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:57:56.462806 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:57:56.462806 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:57:56.462806 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 18 04:57:56.462806 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 18 04:57:56.462806 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 18 04:57:56.462806 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 18 04:57:56.463853 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 18 04:57:56.463853 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 18 04:57:56.463853 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 18 04:57:56.463853 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 18 04:57:56.463853 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:57:56.463853 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:57:56.463853 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:57:56.463853 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:57:56.463853 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:57:56.463853 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:57:56.463853 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:57:56.463853 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jun 18 04:57:56.463853 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:57:56.463853 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:57:56.463853 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:57:56.463853 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:57:56.464603 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:57:56.464603 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:57:56.464603 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:57:56.464603 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:57:56.464603 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 18 04:57:56.464603 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 18 04:57:56.464603 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 18 04:57:56.464603 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 18 04:57:56.464603 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 18 04:57:56.464603 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 18 04:57:56.464603 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 18 04:57:56.464603 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 04:57:56.464603 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 34/70] 10.222.0.23 () {56 vars in 1096 bytes} [Sat Jun 18 04:57:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:57:56.465930 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29c610c-7da6-436c-8f0a-21d08449614a None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:57:56.465930 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29c610c-7da6-436c-8f0a-21d08449614a None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:57:56.465930 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29c610c-7da6-436c-8f0a-21d08449614a None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:57:56.467291 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 35/71] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 04:57:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 04:57:56.467995 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29c610c-7da6-436c-8f0a-21d08449614a None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:57:56.477158 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346785ab-bae2-4549-b76e-1571112d6c26 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:57:56.512936 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd906617-0efa-4da8-be9d-d997b26a2141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:57:56.513778 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd906617-0efa-4da8-be9d-d997b26a2141 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:57:56.514063 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd906617-0efa-4da8-be9d-d997b26a2141 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:57:56.514335 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd906617-0efa-4da8-be9d-d997b26a2141 None admin] PATH_INFO: `/v3/users/21ca63be55244b159bcc68b518233c91` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:57:56.529985 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd906617-0efa-4da8-be9d-d997b26a2141 None admin] RBAC: Authorizing `identity:get_user(user_id=21ca63be55244b159bcc68b518233c91)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:57:56.531454 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd906617-0efa-4da8-be9d-d997b26a2141 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:57:56.533115 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 37/72] 10.222.0.23 () {56 vars in 1177 bytes} [Sat Jun 18 04:57:56 2022] GET /identity/v3/users/21ca63be55244b159bcc68b518233c91 => generated 275 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:57:56.534345 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a92d1e8-fa4a-4078-9bc7-5290128eb98a None None] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:57:56.534633 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a92d1e8-fa4a-4078-9bc7-5290128eb98a None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:57:56.534912 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a92d1e8-fa4a-4078-9bc7-5290128eb98a None None] PATH_INFO: `/` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:57:56.536135 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 38/73] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 04:57:56 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 04:57:56.536846 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a92d1e8-fa4a-4078-9bc7-5290128eb98a None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:57:56.549449 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:57:56.579046 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29c610c-7da6-436c-8f0a-21d08449614a None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:57:56.613016 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf819be8-8a19-4e49-a6c3-6040ec25f202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:57:56.614111 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf819be8-8a19-4e49-a6c3-6040ec25f202 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:57:56.614399 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf819be8-8a19-4e49-a6c3-6040ec25f202 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:57:56.614678 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf819be8-8a19-4e49-a6c3-6040ec25f202 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:57:56.625273 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf819be8-8a19-4e49-a6c3-6040ec25f202 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 18 04:57:56.633439 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-cf819be8-8a19-4e49-a6c3-6040ec25f202 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 18 04:57:56.633439 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:57:56.633439 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:57:56.633439 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:57:56.633439 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:57:56.633439 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:57:56.633439 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:57:56.633439 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:57:56.633439 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:57:56.633439 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:57:56.633439 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:57:56.633439 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:57:56.633439 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 18 04:57:56.633439 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 18 04:57:56.633439 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 18 04:57:56.633439 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 18 04:57:56.634746 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 18 04:57:56.634746 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 18 04:57:56.634746 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 18 04:57:56.634746 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 18 04:57:56.634746 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:57:56.634746 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:57:56.634746 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:57:56.634746 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:57:56.634746 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:57:56.634746 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:57:56.634746 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:57:56.634746 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jun 18 04:57:56.634746 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:57:56.634746 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:57:56.634746 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:57:56.634746 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:57:56.636035 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:57:56.636035 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:57:56.636035 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:57:56.636035 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:57:56.636035 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 18 04:57:56.636035 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 18 04:57:56.636035 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 18 04:57:56.636035 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 18 04:57:56.636035 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 18 04:57:56.636035 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 18 04:57:56.636035 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 18 04:57:56.636035 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 04:57:56.636035 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 36/74] 10.222.0.23 () {56 vars in 1114 bytes} [Sat Jun 18 04:57:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:57:56.636035 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2c23e0-9933-4824-a1ff-071ad007e718 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:57:56.636895 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2c23e0-9933-4824-a1ff-071ad007e718 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:57:56.636895 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2c23e0-9933-4824-a1ff-071ad007e718 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:57:56.646127 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a92d1e8-fa4a-4078-9bc7-5290128eb98a None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:57:56.652599 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-5c2c23e0-9933-4824-a1ff-071ad007e718 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 04:57:56.653853 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-5c2c23e0-9933-4824-a1ff-071ad007e718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 04:57:56.681714 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfb548d-fde5-42fb-9892-713e58438d9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:57:56.682383 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfb548d-fde5-42fb-9892-713e58438d9e None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:57:56.682668 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfb548d-fde5-42fb-9892-713e58438d9e None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:57:56.682948 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfb548d-fde5-42fb-9892-713e58438d9e None admin] PATH_INFO: `/v3/roles` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:57:56.683701 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfb548d-fde5-42fb-9892-713e58438d9e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:57:56.684019 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfb548d-fde5-42fb-9892-713e58438d9e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:57:56.685404 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfb548d-fde5-42fb-9892-713e58438d9e None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:57:56.693539 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 39/75] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Jun 18 04:57:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:57:56.696124 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfb548d-fde5-42fb-9892-713e58438d9e None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:57:56.708186 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:57:56.721651 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 37/76] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 04:57:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 18 04:57:56.728055 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951ce6c6-e210-4c6d-9ff0-9065c289a792 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:57:56.728351 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951ce6c6-e210-4c6d-9ff0-9065c289a792 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:57:56.728637 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951ce6c6-e210-4c6d-9ff0-9065c289a792 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:57:56.745261 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-951ce6c6-e210-4c6d-9ff0-9065c289a792 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 04:57:56.746513 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-951ce6c6-e210-4c6d-9ff0-9065c289a792 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 04:57:56.769966 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:57:56.800113 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfb548d-fde5-42fb-9892-713e58438d9e None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:57:56.813655 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 38/77] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 04:57:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 18 04:57:56.814478 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951ce6c6-e210-4c6d-9ff0-9065c289a792 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:57:56.839368 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb7ee51-6451-434a-8aa2-b90b3ef9a6ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:57:56.840012 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb7ee51-6451-434a-8aa2-b90b3ef9a6ef None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:57:56.840295 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb7ee51-6451-434a-8aa2-b90b3ef9a6ef None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:57:56.840570 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb7ee51-6451-434a-8aa2-b90b3ef9a6ef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:57:56.909062 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb7ee51-6451-434a-8aa2-b90b3ef9a6ef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:57:56.910560 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb7ee51-6451-434a-8aa2-b90b3ef9a6ef None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:57:56.916298 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951ce6c6-e210-4c6d-9ff0-9065c289a792 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:57:56.933649 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:57:56.950061 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95ad80c-f0fe-4678-a9e6-89e3b48ca764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:57:56.950734 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95ad80c-f0fe-4678-a9e6-89e3b48ca764 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:57:56.951025 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95ad80c-f0fe-4678-a9e6-89e3b48ca764 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:57:56.951311 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95ad80c-f0fe-4678-a9e6-89e3b48ca764 None admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:57:56.952066 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95ad80c-f0fe-4678-a9e6-89e3b48ca764 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:57:56.952383 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95ad80c-f0fe-4678-a9e6-89e3b48ca764 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:57:56.953827 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95ad80c-f0fe-4678-a9e6-89e3b48ca764 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:57:56.967576 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 39/78] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Jun 18 04:57:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:57:56.967576 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95ad80c-f0fe-4678-a9e6-89e3b48ca764 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:57:56.989668 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:57:57.020312 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 40/79] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Jun 18 04:57:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 18 04:57:57.026346 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb7ee51-6451-434a-8aa2-b90b3ef9a6ef None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:57:57.067052 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95ad80c-f0fe-4678-a9e6-89e3b48ca764 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:57:57.102834 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ce9938-2f34-41ec-bdc4-64ed8a079352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:57:57.103700 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ce9938-2f34-41ec-bdc4-64ed8a079352 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:57:57.104063 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ce9938-2f34-41ec-bdc4-64ed8a079352 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:57:57.104267 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ce9938-2f34-41ec-bdc4-64ed8a079352 None admin] PATH_INFO: `/v3/users/5d3e83f3ca7347dea1350fa29bac5d9d` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:57:57.105477 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ce9938-2f34-41ec-bdc4-64ed8a079352 None admin] RBAC: Authorizing `identity:get_user(user_id=5d3e83f3ca7347dea1350fa29bac5d9d)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:57:57.106946 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ce9938-2f34-41ec-bdc4-64ed8a079352 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:57:57.108747 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 40/80] 10.222.0.23 () {56 vars in 1177 bytes} [Sat Jun 18 04:57:56 2022] GET /identity/v3/users/5d3e83f3ca7347dea1350fa29bac5d9d => generated 247 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:57:57.110216 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ce9938-2f34-41ec-bdc4-64ed8a079352 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:57:57.125053 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb7ee51-6451-434a-8aa2-b90b3ef9a6ef None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:57:57.157949 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:57:57.159238 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c981e1ac-2697-4a68-a143-f3cd38b8d254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:57:57.160012 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c981e1ac-2697-4a68-a143-f3cd38b8d254 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:57:57.160304 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c981e1ac-2697-4a68-a143-f3cd38b8d254 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:57:57.160588 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c981e1ac-2697-4a68-a143-f3cd38b8d254 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:57:57.170825 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c981e1ac-2697-4a68-a143-f3cd38b8d254 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 18 04:57:57.179312 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application [None req-c981e1ac-2697-4a68-a143-f3cd38b8d254 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 18 04:57:57.179312 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:57:57.179312 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:57:57.179312 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:57:57.179312 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:57:57.179312 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:57:57.179312 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:57:57.179312 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:57:57.179312 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:57:57.179312 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:57:57.179312 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:57:57.179312 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:57:57.179312 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 18 04:57:57.179312 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 18 04:57:57.179312 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 18 04:57:57.179312 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 18 04:57:57.180195 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 18 04:57:57.180195 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 18 04:57:57.180195 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 18 04:57:57.180195 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 18 04:57:57.180195 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:57:57.180195 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:57:57.180195 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:57:57.180195 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:57:57.180195 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:57:57.180195 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:57:57.180195 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:57:57.180195 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application with Lock( Jun 18 04:57:57.180195 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:57:57.180195 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:57:57.180195 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:57:57.180195 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:57:57.180967 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:57:57.180967 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:57:57.180967 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:57:57.180967 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:57:57.180967 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 18 04:57:57.180967 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 18 04:57:57.180967 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 18 04:57:57.180967 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 18 04:57:57.180967 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 18 04:57:57.180967 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 18 04:57:57.180967 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 18 04:57:57.180967 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 04:57:57.180967 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 41/81] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Jun 18 04:57:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:57:57.181628 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c981e1ac-2697-4a68-a143-f3cd38b8d254 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:57:57.209037 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ce9938-2f34-41ec-bdc4-64ed8a079352 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:57:57.216613 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:57:57.243878 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e18a539-7bb5-49c1-8623-1ace9d2149a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:57:57.244674 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e18a539-7bb5-49c1-8623-1ace9d2149a7 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:57:57.244961 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e18a539-7bb5-49c1-8623-1ace9d2149a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:57:57.245241 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e18a539-7bb5-49c1-8623-1ace9d2149a7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:57:57.246459 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e18a539-7bb5-49c1-8623-1ace9d2149a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:57:57.247912 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e18a539-7bb5-49c1-8623-1ace9d2149a7 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:57:57.249580 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 41/82] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Jun 18 04:57:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:57:57.249580 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e18a539-7bb5-49c1-8623-1ace9d2149a7 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:57:57.287585 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c981e1ac-2697-4a68-a143-f3cd38b8d254 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:57:57.321268 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47db94e-e5a6-41f1-adc1-562a2de28789 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:57:57.322162 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47db94e-e5a6-41f1-adc1-562a2de28789 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:57:57.322452 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47db94e-e5a6-41f1-adc1-562a2de28789 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:57:57.322733 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47db94e-e5a6-41f1-adc1-562a2de28789 None admin] PATH_INFO: `/v3/users/21ca63be55244b159bcc68b518233c91` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:57:57.323983 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47db94e-e5a6-41f1-adc1-562a2de28789 None admin] RBAC: Authorizing `identity:get_user(user_id=21ca63be55244b159bcc68b518233c91)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:57:57.325481 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47db94e-e5a6-41f1-adc1-562a2de28789 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:57:57.327124 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 42/83] 10.222.0.23 () {56 vars in 1177 bytes} [Sat Jun 18 04:57:57 2022] GET /identity/v3/users/21ca63be55244b159bcc68b518233c91 => generated 275 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:57:57.327897 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47db94e-e5a6-41f1-adc1-562a2de28789 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:57:57.359032 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e18a539-7bb5-49c1-8623-1ace9d2149a7 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:57:57.375784 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:57:57.394725 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e408878b-237a-4008-86b5-f70018a13346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:57:57.395636 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e408878b-237a-4008-86b5-f70018a13346 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:57:57.395923 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e408878b-237a-4008-86b5-f70018a13346 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:57:57.396205 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e408878b-237a-4008-86b5-f70018a13346 None admin] PATH_INFO: `/v3/projects/6706fdb556af472c8319dd2a13abe441` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:57:57.397981 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e408878b-237a-4008-86b5-f70018a13346 None admin] RBAC: Authorizing `identity:get_project(project_id=6706fdb556af472c8319dd2a13abe441)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:57:57.399477 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e408878b-237a-4008-86b5-f70018a13346 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:57:57.401254 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 42/84] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Jun 18 04:57:57 2022] GET /identity/v3/projects/6706fdb556af472c8319dd2a13abe441 => generated 298 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:57:57.402049 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e408878b-237a-4008-86b5-f70018a13346 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:57:57.427939 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:57:57.431124 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47db94e-e5a6-41f1-adc1-562a2de28789 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:57:57.464520 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf28a9fb-1ee6-4f69-bf2f-97dd6f147cbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:57:57.465337 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf28a9fb-1ee6-4f69-bf2f-97dd6f147cbe None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:57:57.465925 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf28a9fb-1ee6-4f69-bf2f-97dd6f147cbe None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:57:57.466212 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf28a9fb-1ee6-4f69-bf2f-97dd6f147cbe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:57:57.502382 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e408878b-237a-4008-86b5-f70018a13346 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:57:57.533213 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf28a9fb-1ee6-4f69-bf2f-97dd6f147cbe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:57:57.534713 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf28a9fb-1ee6-4f69-bf2f-97dd6f147cbe None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:57:57.537073 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114d7ef9-eaef-4e1e-a1d3-2d179c9bf439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:57:57.537899 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114d7ef9-eaef-4e1e-a1d3-2d179c9bf439 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:57:57.538160 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114d7ef9-eaef-4e1e-a1d3-2d179c9bf439 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:57:57.538468 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114d7ef9-eaef-4e1e-a1d3-2d179c9bf439 None admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:57:57.539220 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114d7ef9-eaef-4e1e-a1d3-2d179c9bf439 None admin] RBAC: Adding query filter params (name=member) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:57:57.539526 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114d7ef9-eaef-4e1e-a1d3-2d179c9bf439 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:57:57.540891 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114d7ef9-eaef-4e1e-a1d3-2d179c9bf439 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:57:57.548934 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 43/85] 10.222.0.23 () {56 vars in 1101 bytes} [Sat Jun 18 04:57:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:57:57.549732 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114d7ef9-eaef-4e1e-a1d3-2d179c9bf439 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:57:57.595244 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:57:57.640112 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:57:57.701452 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 43/86] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Jun 18 04:57:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 18 04:57:57.701452 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf28a9fb-1ee6-4f69-bf2f-97dd6f147cbe None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:57:57.702123 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114d7ef9-eaef-4e1e-a1d3-2d179c9bf439 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:57:57.702123 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176e36c8-eed2-4e78-8384-ead699e36c2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:57:57.702123 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176e36c8-eed2-4e78-8384-ead699e36c2e None admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:57:57.702123 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176e36c8-eed2-4e78-8384-ead699e36c2e None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:57:57.702744 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176e36c8-eed2-4e78-8384-ead699e36c2e None admin] PATH_INFO: `/v3/users` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:57:57.702744 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176e36c8-eed2-4e78-8384-ead699e36c2e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:57:57.702744 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176e36c8-eed2-4e78-8384-ead699e36c2e None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:57:57.744649 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf28a9fb-1ee6-4f69-bf2f-97dd6f147cbe None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:57:57.777544 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd3468e-0992-4461-a234-bcaa3c6fc996 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:57:57.778178 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd3468e-0992-4461-a234-bcaa3c6fc996 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:57:57.778458 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd3468e-0992-4461-a234-bcaa3c6fc996 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:57:57.778730 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd3468e-0992-4461-a234-bcaa3c6fc996 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:57:57.795341 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 44/87] 10.222.0.23 () {60 vars in 1132 bytes} [Sat Jun 18 04:57:57 2022] POST /identity/v3/users => generated 289 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 04:57:57.804257 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176e36c8-eed2-4e78-8384-ead699e36c2e None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:57:57.852842 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:57:57.855937 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd3468e-0992-4461-a234-bcaa3c6fc996 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:57:57.857720 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd3468e-0992-4461-a234-bcaa3c6fc996 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:57:57.917864 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176e36c8-eed2-4e78-8384-ead699e36c2e None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:57:57.919810 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:57:57.963309 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f3940e-ab96-4728-a20c-2307a7dc2d2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:57:57.963974 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f3940e-ab96-4728-a20c-2307a7dc2d2c None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:57:57.964262 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f3940e-ab96-4728-a20c-2307a7dc2d2c None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:57:57.964539 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f3940e-ab96-4728-a20c-2307a7dc2d2c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:57:57.965369 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f3940e-ab96-4728-a20c-2307a7dc2d2c None admin] RBAC: Adding query filter params (role.id=cbf5c4c26898413d93e73b46b41046a3, scope.project.id=6706fdb556af472c8319dd2a13abe441, user.id=21ca63be55244b159bcc68b518233c91) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:57:57.967278 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f3940e-ab96-4728-a20c-2307a7dc2d2c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:57:57.967827 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f3940e-ab96-4728-a20c-2307a7dc2d2c None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:57:57.973581 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 45/88] 10.222.0.23 () {56 vars in 1374 bytes} [Sat Jun 18 04:57:57 2022] GET /identity/v3/role_assignments?user.id=21ca63be55244b159bcc68b518233c91&scope.project.id=6706fdb556af472c8319dd2a13abe441&role.id=cbf5c4c26898413d93e73b46b41046a3 => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:57:57.974382 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f3940e-ab96-4728-a20c-2307a7dc2d2c None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:57:57.977363 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 44/89] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Jun 18 04:57:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 18 04:57:57.978267 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd3468e-0992-4461-a234-bcaa3c6fc996 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:57:58.129120 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f3940e-ab96-4728-a20c-2307a7dc2d2c None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:57:58.138061 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd3468e-0992-4461-a234-bcaa3c6fc996 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:57:58.139995 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:57:58.167606 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a2f195-9443-4441-a6bd-6ad08a6e0c4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:57:58.171125 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a2f195-9443-4441-a6bd-6ad08a6e0c4f None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:57:58.171125 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a2f195-9443-4441-a6bd-6ad08a6e0c4f None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:57:58.171125 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a2f195-9443-4441-a6bd-6ad08a6e0c4f None admin] PATH_INFO: `/v3/users/21ca63be55244b159bcc68b518233c91` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:57:58.171125 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a2f195-9443-4441-a6bd-6ad08a6e0c4f None admin] RBAC: Authorizing `identity:get_user(user_id=21ca63be55244b159bcc68b518233c91)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:57:58.171681 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a2f195-9443-4441-a6bd-6ad08a6e0c4f None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:57:58.173361 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 46/90] 10.222.0.23 () {56 vars in 1177 bytes} [Sat Jun 18 04:57:57 2022] GET /identity/v3/users/21ca63be55244b159bcc68b518233c91 => generated 275 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:57:58.174882 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622d4d07-cbf1-4cb0-8bdc-1acf57c8ea25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:57:58.175680 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a2f195-9443-4441-a6bd-6ad08a6e0c4f None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:57:58.175966 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622d4d07-cbf1-4cb0-8bdc-1acf57c8ea25 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:57:58.176255 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622d4d07-cbf1-4cb0-8bdc-1acf57c8ea25 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:57:58.176533 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622d4d07-cbf1-4cb0-8bdc-1acf57c8ea25 None admin] PATH_INFO: `/v3/projects/6706fdb556af472c8319dd2a13abe441` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:57:58.179876 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622d4d07-cbf1-4cb0-8bdc-1acf57c8ea25 None admin] RBAC: Authorizing `identity:get_project(project_id=6706fdb556af472c8319dd2a13abe441)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:57:58.181371 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622d4d07-cbf1-4cb0-8bdc-1acf57c8ea25 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:57:58.183181 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 45/91] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Jun 18 04:57:57 2022] GET /identity/v3/projects/6706fdb556af472c8319dd2a13abe441 => generated 298 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:57:58.183955 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622d4d07-cbf1-4cb0-8bdc-1acf57c8ea25 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:57:58.228286 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:57:58.339413 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a2f195-9443-4441-a6bd-6ad08a6e0c4f None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:57:58.356845 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622d4d07-cbf1-4cb0-8bdc-1acf57c8ea25 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:57:58.411052 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f275710-a55a-4334-9b98-5fb3a0d233c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:57:58.411931 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f275710-a55a-4334-9b98-5fb3a0d233c7 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:57:58.412223 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f275710-a55a-4334-9b98-5fb3a0d233c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:57:58.413025 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f275710-a55a-4334-9b98-5fb3a0d233c7 None admin] PATH_INFO: `/v3/projects/6706fdb556af472c8319dd2a13abe441` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:57:58.414601 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f275710-a55a-4334-9b98-5fb3a0d233c7 None admin] RBAC: Authorizing `identity:get_project(project_id=6706fdb556af472c8319dd2a13abe441)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:57:58.416072 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f275710-a55a-4334-9b98-5fb3a0d233c7 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:57:58.417845 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 47/92] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Jun 18 04:57:58 2022] GET /identity/v3/projects/6706fdb556af472c8319dd2a13abe441 => generated 298 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:57:58.418622 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f275710-a55a-4334-9b98-5fb3a0d233c7 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:57:58.421888 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a450dbe2-40b5-41a9-a25a-52b5e327654b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:57:58.422601 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a450dbe2-40b5-41a9-a25a-52b5e327654b None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:57:58.422899 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a450dbe2-40b5-41a9-a25a-52b5e327654b None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:57:58.423183 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a450dbe2-40b5-41a9-a25a-52b5e327654b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:57:58.495097 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:57:58.554480 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f275710-a55a-4334-9b98-5fb3a0d233c7 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:57:58.556094 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:57:58.566476 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a450dbe2-40b5-41a9-a25a-52b5e327654b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:57:58.568109 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a450dbe2-40b5-41a9-a25a-52b5e327654b None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:57:58.595048 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024f367f-8f4b-447a-88b1-19ded72f63fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:57:58.595721 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024f367f-8f4b-447a-88b1-19ded72f63fa None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:57:58.596013 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024f367f-8f4b-447a-88b1-19ded72f63fa None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:57:58.596298 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024f367f-8f4b-447a-88b1-19ded72f63fa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:57:58.597142 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024f367f-8f4b-447a-88b1-19ded72f63fa None admin] RBAC: Adding query filter params (role.id=21c65f2c51bd4dd283d13507d2714467, scope.project.id=6706fdb556af472c8319dd2a13abe441, user.id=5d3e83f3ca7347dea1350fa29bac5d9d) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:57:58.597528 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024f367f-8f4b-447a-88b1-19ded72f63fa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:57:58.599031 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024f367f-8f4b-447a-88b1-19ded72f63fa None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:57:58.617767 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 48/93] 10.222.0.23 () {56 vars in 1374 bytes} [Sat Jun 18 04:57:58 2022] GET /identity/v3/role_assignments?user.id=5d3e83f3ca7347dea1350fa29bac5d9d&scope.project.id=6706fdb556af472c8319dd2a13abe441&role.id=21c65f2c51bd4dd283d13507d2714467 => generated 259 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:57:58.617767 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024f367f-8f4b-447a-88b1-19ded72f63fa None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:57:58.752446 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 46/94] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Jun 18 04:57:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 569 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 18 04:57:58.758451 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a450dbe2-40b5-41a9-a25a-52b5e327654b None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:57:58.880583 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:57:58.904590 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024f367f-8f4b-447a-88b1-19ded72f63fa None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:57:58.939305 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a450dbe2-40b5-41a9-a25a-52b5e327654b None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:57:58.970093 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717ab7c7-3f3c-4217-ba07-9a593bd0e795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:57:58.970759 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717ab7c7-3f3c-4217-ba07-9a593bd0e795 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:57:58.971049 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717ab7c7-3f3c-4217-ba07-9a593bd0e795 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:57:58.971333 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717ab7c7-3f3c-4217-ba07-9a593bd0e795 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:57:58.972175 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717ab7c7-3f3c-4217-ba07-9a593bd0e795 None admin] RBAC: Adding query filter params (role.id=0089a3aad83c4ae08d8e9d864957285b, scope.project.id=6706fdb556af472c8319dd2a13abe441, user.id=21ca63be55244b159bcc68b518233c91) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:57:58.972497 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717ab7c7-3f3c-4217-ba07-9a593bd0e795 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:57:58.973991 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717ab7c7-3f3c-4217-ba07-9a593bd0e795 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:57:58.983235 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 49/95] 10.222.0.23 () {56 vars in 1374 bytes} [Sat Jun 18 04:57:58 2022] GET /identity/v3/role_assignments?user.id=21ca63be55244b159bcc68b518233c91&scope.project.id=6706fdb556af472c8319dd2a13abe441&role.id=0089a3aad83c4ae08d8e9d864957285b => generated 259 bytes in 369 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:57:58.995687 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947ef2e9-dc09-4b65-9c75-74c6dfd179ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:57:58.996568 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947ef2e9-dc09-4b65-9c75-74c6dfd179ad None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:57:58.996859 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947ef2e9-dc09-4b65-9c75-74c6dfd179ad None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:57:58.997141 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947ef2e9-dc09-4b65-9c75-74c6dfd179ad None admin] PATH_INFO: `/v3/projects/fa4a0c00884f4f97b9e8fc0496982551` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:57:58.998421 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947ef2e9-dc09-4b65-9c75-74c6dfd179ad None admin] RBAC: Authorizing `identity:get_project(project_id=fa4a0c00884f4f97b9e8fc0496982551)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:57:58.999937 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947ef2e9-dc09-4b65-9c75-74c6dfd179ad None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:57:59.002059 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 47/96] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Jun 18 04:57:58 2022] GET /identity/v3/projects/fa4a0c00884f4f97b9e8fc0496982551 => generated 312 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:57:59.008549 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717ab7c7-3f3c-4217-ba07-9a593bd0e795 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:57:59.055639 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:57:59.156724 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717ab7c7-3f3c-4217-ba07-9a593bd0e795 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:57:59.204549 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2ce67a-1c7a-4548-baa9-6ef7bb878eb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:57:59.205221 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2ce67a-1c7a-4548-baa9-6ef7bb878eb6 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:57:59.205537 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2ce67a-1c7a-4548-baa9-6ef7bb878eb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:57:59.205820 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2ce67a-1c7a-4548-baa9-6ef7bb878eb6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:57:59.206661 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2ce67a-1c7a-4548-baa9-6ef7bb878eb6 None admin] RBAC: Adding query filter params (role.id=cbf5c4c26898413d93e73b46b41046a3, scope.project.id=fa4a0c00884f4f97b9e8fc0496982551, user.id=21ca63be55244b159bcc68b518233c91) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:57:59.206984 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2ce67a-1c7a-4548-baa9-6ef7bb878eb6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:57:59.208444 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2ce67a-1c7a-4548-baa9-6ef7bb878eb6 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:57:59.221567 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 50/97] 10.222.0.23 () {56 vars in 1374 bytes} [Sat Jun 18 04:57:59 2022] GET /identity/v3/role_assignments?user.id=21ca63be55244b159bcc68b518233c91&scope.project.id=fa4a0c00884f4f97b9e8fc0496982551&role.id=cbf5c4c26898413d93e73b46b41046a3 => generated 259 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:00.791497 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b58e056-7cf9-4b5f-8f2f-dc0ca6a3409b None None] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:00.792002 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b58e056-7cf9-4b5f-8f2f-dc0ca6a3409b None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:00.792101 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b58e056-7cf9-4b5f-8f2f-dc0ca6a3409b None None] PATH_INFO: `/` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:00.793699 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 48/98] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 04:58:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 04:58:00.798323 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b85b66-5c69-4238-9ef7-5a23b06753b7 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:00.799063 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f37487-8b58-441d-83bc-f08f3dc212fa None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:00.799363 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f37487-8b58-441d-83bc-f08f3dc212fa None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:00.799565 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b85b66-5c69-4238-9ef7-5a23b06753b7 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:00.799659 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f37487-8b58-441d-83bc-f08f3dc212fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:00.800141 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b85b66-5c69-4238-9ef7-5a23b06753b7 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:00.801856 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 51/99] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 04:58:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 04:58:00.807417 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc2f845-c77c-40d4-a7bc-09dce6da8a9b None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:00.807955 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc2f845-c77c-40d4-a7bc-09dce6da8a9b None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:00.808433 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc2f845-c77c-40d4-a7bc-09dce6da8a9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:00.820214 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-36f37487-8b58-441d-83bc-f08f3dc212fa None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 04:58:00.821445 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-36f37487-8b58-441d-83bc-f08f3dc212fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 04:58:00.841563 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-2bc2f845-c77c-40d4-a7bc-09dce6da8a9b None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 04:58:00.842779 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-2bc2f845-c77c-40d4-a7bc-09dce6da8a9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 04:58:00.936922 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 49/100] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 04:58:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 18 04:58:00.976004 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 52/101] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 04:58:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 18 04:58:00.979338 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f37487-8b58-441d-83bc-f08f3dc212fa None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:01.046611 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc2f845-c77c-40d4-a7bc-09dce6da8a9b None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:01.097753 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:01.101741 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:01.170734 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f37487-8b58-441d-83bc-f08f3dc212fa None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:01.242810 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f745a4c-292e-4470-aec3-60246e19bdf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:01.243615 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f745a4c-292e-4470-aec3-60246e19bdf9 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:01.243904 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f745a4c-292e-4470-aec3-60246e19bdf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:01.244183 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f745a4c-292e-4470-aec3-60246e19bdf9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:01.262006 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc2f845-c77c-40d4-a7bc-09dce6da8a9b None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:01.273547 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f745a4c-292e-4470-aec3-60246e19bdf9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 18 04:58:01.283247 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application [None req-8f745a4c-292e-4470-aec3-60246e19bdf9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 18 04:58:01.283247 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:58:01.283247 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:58:01.283247 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:58:01.283247 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:58:01.283247 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:58:01.283247 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:58:01.283247 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:58:01.283247 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:58:01.283247 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:58:01.283247 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:58:01.283247 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:58:01.283247 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 18 04:58:01.283247 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 18 04:58:01.283247 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 18 04:58:01.283247 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 18 04:58:01.284117 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 18 04:58:01.284117 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 18 04:58:01.284117 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 18 04:58:01.284117 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 18 04:58:01.284117 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:58:01.284117 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:58:01.284117 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:58:01.284117 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:58:01.284117 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:58:01.284117 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:58:01.284117 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:58:01.284117 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application with Lock( Jun 18 04:58:01.284117 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:58:01.284117 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:58:01.284117 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:58:01.284117 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:58:01.284985 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:58:01.284985 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:58:01.284985 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:58:01.284985 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:58:01.284985 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 18 04:58:01.284985 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 18 04:58:01.284985 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 18 04:58:01.284985 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 18 04:58:01.284985 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 18 04:58:01.284985 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 18 04:58:01.284985 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 18 04:58:01.284985 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 04:58:01.286704 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 50/102] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Jun 18 04:58:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:58:01.291609 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f745a4c-292e-4470-aec3-60246e19bdf9 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:01.348307 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad54e6c-9c77-4ec7-b82a-4a8e24e320e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:01.349038 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad54e6c-9c77-4ec7-b82a-4a8e24e320e7 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:01.349327 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad54e6c-9c77-4ec7-b82a-4a8e24e320e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:01.349650 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad54e6c-9c77-4ec7-b82a-4a8e24e320e7 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:01.376327 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad54e6c-9c77-4ec7-b82a-4a8e24e320e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 18 04:58:01.382933 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-6ad54e6c-9c77-4ec7-b82a-4a8e24e320e7 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 18 04:58:01.382933 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:58:01.382933 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:58:01.382933 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:58:01.382933 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:58:01.382933 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:58:01.382933 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:58:01.382933 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:58:01.382933 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:58:01.382933 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:58:01.382933 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:58:01.382933 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:58:01.382933 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 18 04:58:01.382933 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 18 04:58:01.382933 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 18 04:58:01.382933 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 18 04:58:01.383771 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 18 04:58:01.383771 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 18 04:58:01.383771 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 18 04:58:01.383771 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 18 04:58:01.383771 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:58:01.383771 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:58:01.383771 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:58:01.383771 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:58:01.383771 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:58:01.383771 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:58:01.383771 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:58:01.383771 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jun 18 04:58:01.383771 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:58:01.383771 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:58:01.383771 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:58:01.383771 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:58:01.384521 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:58:01.384521 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:58:01.384521 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:58:01.384521 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:58:01.384521 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 18 04:58:01.384521 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 18 04:58:01.384521 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 18 04:58:01.384521 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 18 04:58:01.384521 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 18 04:58:01.384521 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 18 04:58:01.384521 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 18 04:58:01.384521 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 04:58:01.384521 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 53/103] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Jun 18 04:58:01 2022] GET /identity/v3/roles/reader => generated 84 bytes in 338 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 18 04:58:01.385974 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8f588e-c686-4cfc-ad47-d945df594fa9 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:01.398122 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8f588e-c686-4cfc-ad47-d945df594fa9 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:01.398418 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8f588e-c686-4cfc-ad47-d945df594fa9 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:01.399928 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 54/104] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 04:58:01 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 04:58:01.400693 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8f588e-c686-4cfc-ad47-d945df594fa9 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:01.505138 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f745a4c-292e-4470-aec3-60246e19bdf9 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:01.557772 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:01.570299 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:01.578081 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa777e1-923e-4613-ad20-9ec4e58b0877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:01.579565 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa777e1-923e-4613-ad20-9ec4e58b0877 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:01.579565 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa777e1-923e-4613-ad20-9ec4e58b0877 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:01.579565 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa777e1-923e-4613-ad20-9ec4e58b0877 None admin] PATH_INFO: `/v3/roles` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:01.580115 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa777e1-923e-4613-ad20-9ec4e58b0877 None admin] RBAC: Adding query filter params (name=member) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:58:01.580514 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa777e1-923e-4613-ad20-9ec4e58b0877 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:01.581958 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa777e1-923e-4613-ad20-9ec4e58b0877 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:01.591518 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8f588e-c686-4cfc-ad47-d945df594fa9 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:01.607855 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 51/105] 10.222.0.23 () {56 vars in 1101 bytes} [Sat Jun 18 04:58:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:01.609534 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0379b8-7715-4cc9-9016-fc96a2754bd7 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:01.610025 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0379b8-7715-4cc9-9016-fc96a2754bd7 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:01.610501 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0379b8-7715-4cc9-9016-fc96a2754bd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:01.638225 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df977b1d-ea6a-4307-ad37-4856deb5c260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:01.638912 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df977b1d-ea6a-4307-ad37-4856deb5c260 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:01.639202 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df977b1d-ea6a-4307-ad37-4856deb5c260 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:01.639483 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df977b1d-ea6a-4307-ad37-4856deb5c260 None admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:01.640246 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df977b1d-ea6a-4307-ad37-4856deb5c260 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:58:01.640560 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df977b1d-ea6a-4307-ad37-4856deb5c260 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:01.642016 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df977b1d-ea6a-4307-ad37-4856deb5c260 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:01.645319 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-9f0379b8-7715-4cc9-9016-fc96a2754bd7 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 04:58:01.646573 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-9f0379b8-7715-4cc9-9016-fc96a2754bd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 04:58:01.651746 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 55/106] 10.222.0.23 () {56 vars in 1101 bytes} [Sat Jun 18 04:58:01 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:01.652744 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df977b1d-ea6a-4307-ad37-4856deb5c260 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:01.744209 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 52/107] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 04:58:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 18 04:58:01.745905 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a251296-d2a5-426f-a043-29179a836262 None None] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:01.746397 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a251296-d2a5-426f-a043-29179a836262 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:01.746902 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a251296-d2a5-426f-a043-29179a836262 None None] PATH_INFO: `/` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:01.748367 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 53/108] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 04:58:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 04:58:01.753136 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a251296-d2a5-426f-a043-29179a836262 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:01.802709 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df977b1d-ea6a-4307-ad37-4856deb5c260 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:01.842461 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4bd40e-c10c-46cd-a2ad-f52d0ae44a0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:01.843352 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4bd40e-c10c-46cd-a2ad-f52d0ae44a0b None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:01.843705 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4bd40e-c10c-46cd-a2ad-f52d0ae44a0b None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:01.843996 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4bd40e-c10c-46cd-a2ad-f52d0ae44a0b None admin] PATH_INFO: `/v3/users/21ca63be55244b159bcc68b518233c91` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:01.845219 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4bd40e-c10c-46cd-a2ad-f52d0ae44a0b None admin] RBAC: Authorizing `identity:get_user(user_id=21ca63be55244b159bcc68b518233c91)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:01.846709 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4bd40e-c10c-46cd-a2ad-f52d0ae44a0b None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:01.848530 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 56/109] 10.222.0.23 () {56 vars in 1177 bytes} [Sat Jun 18 04:58:01 2022] GET /identity/v3/users/21ca63be55244b159bcc68b518233c91 => generated 275 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:01.849648 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4c7744-6cba-4e77-82c1-e5aa4a2c83e1 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:01.850724 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4c7744-6cba-4e77-82c1-e5aa4a2c83e1 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:01.851211 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4c7744-6cba-4e77-82c1-e5aa4a2c83e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:01.869769 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-1b4c7744-6cba-4e77-82c1-e5aa4a2c83e1 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 04:58:01.870922 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-1b4c7744-6cba-4e77-82c1-e5aa4a2c83e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 04:58:01.873101 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a251296-d2a5-426f-a043-29179a836262 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:01.880605 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:01.893891 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:01.912903 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ba5193-888c-4562-a1d1-dbccc733434a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:01.913824 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ba5193-888c-4562-a1d1-dbccc733434a None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:01.914114 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ba5193-888c-4562-a1d1-dbccc733434a None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:01.914396 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ba5193-888c-4562-a1d1-dbccc733434a None admin] PATH_INFO: `/v3/users/2b6fc67a02584c8cac37b9b3f2205270` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:01.930928 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ba5193-888c-4562-a1d1-dbccc733434a None admin] RBAC: Authorizing `identity:get_user(user_id=2b6fc67a02584c8cac37b9b3f2205270)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:01.932442 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ba5193-888c-4562-a1d1-dbccc733434a None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:01.938516 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 54/110] 10.222.0.23 () {56 vars in 1177 bytes} [Sat Jun 18 04:58:01 2022] GET /identity/v3/users/2b6fc67a02584c8cac37b9b3f2205270 => generated 289 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:01.938516 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ba5193-888c-4562-a1d1-dbccc733434a None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:01.949809 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 57/111] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 04:58:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 18 04:58:01.951210 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74928379-418e-415a-afa4-94c6413c83a9 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:01.951504 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74928379-418e-415a-afa4-94c6413c83a9 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:01.951810 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74928379-418e-415a-afa4-94c6413c83a9 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:01.953400 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 58/112] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 04:58:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 04:58:01.954001 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74928379-418e-415a-afa4-94c6413c83a9 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:02.057950 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ba5193-888c-4562-a1d1-dbccc733434a None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:02.068275 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74928379-418e-415a-afa4-94c6413c83a9 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:02.094059 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54152602-7ae5-43b8-9acc-32bf2a0fd371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:02.094836 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54152602-7ae5-43b8-9acc-32bf2a0fd371 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:02.095126 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54152602-7ae5-43b8-9acc-32bf2a0fd371 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:02.095407 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54152602-7ae5-43b8-9acc-32bf2a0fd371 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:02.096596 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54152602-7ae5-43b8-9acc-32bf2a0fd371 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:02.098108 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54152602-7ae5-43b8-9acc-32bf2a0fd371 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:02.099743 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 55/113] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Jun 18 04:58:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:02.100532 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54152602-7ae5-43b8-9acc-32bf2a0fd371 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:02.108116 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f91177-52bd-49ee-80d7-1175300c4a22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:02.108985 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f91177-52bd-49ee-80d7-1175300c4a22 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:02.109269 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f91177-52bd-49ee-80d7-1175300c4a22 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:02.109587 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f91177-52bd-49ee-80d7-1175300c4a22 None admin] PATH_INFO: `/v3/projects/6706fdb556af472c8319dd2a13abe441` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:02.110792 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f91177-52bd-49ee-80d7-1175300c4a22 None admin] RBAC: Authorizing `identity:get_project(project_id=6706fdb556af472c8319dd2a13abe441)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:02.112258 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f91177-52bd-49ee-80d7-1175300c4a22 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:02.114261 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 59/114] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Jun 18 04:58:01 2022] GET /identity/v3/projects/6706fdb556af472c8319dd2a13abe441 => generated 298 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:02.115445 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06378f83-7fd7-409d-956a-af8475a56271 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:02.116147 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06378f83-7fd7-409d-956a-af8475a56271 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:02.116147 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06378f83-7fd7-409d-956a-af8475a56271 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:02.132732 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-06378f83-7fd7-409d-956a-af8475a56271 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 04:58:02.133894 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-06378f83-7fd7-409d-956a-af8475a56271 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 04:58:02.141004 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:02.145908 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:02.199890 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54152602-7ae5-43b8-9acc-32bf2a0fd371 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:02.202274 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 60/115] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 04:58:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 18 04:58:02.203283 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06378f83-7fd7-409d-956a-af8475a56271 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:02.240228 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd44ee4b-cd6d-426e-a57f-e9feb9e6eae4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:02.241459 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd44ee4b-cd6d-426e-a57f-e9feb9e6eae4 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:02.241751 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd44ee4b-cd6d-426e-a57f-e9feb9e6eae4 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:02.242027 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd44ee4b-cd6d-426e-a57f-e9feb9e6eae4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:02.304705 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06378f83-7fd7-409d-956a-af8475a56271 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:02.308753 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd44ee4b-cd6d-426e-a57f-e9feb9e6eae4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:02.310259 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd44ee4b-cd6d-426e-a57f-e9feb9e6eae4 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:02.338815 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4113dd-5fa2-449b-8e64-7e4966f8c309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:02.339614 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4113dd-5fa2-449b-8e64-7e4966f8c309 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:02.339901 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4113dd-5fa2-449b-8e64-7e4966f8c309 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:02.340185 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4113dd-5fa2-449b-8e64-7e4966f8c309 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:02.350203 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4113dd-5fa2-449b-8e64-7e4966f8c309 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 18 04:58:02.354467 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:02.357949 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-4f4113dd-5fa2-449b-8e64-7e4966f8c309 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 18 04:58:02.357949 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:58:02.357949 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:58:02.357949 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:58:02.357949 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:58:02.357949 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:58:02.357949 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:58:02.357949 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:58:02.357949 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:58:02.357949 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:58:02.357949 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:58:02.357949 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:58:02.357949 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 18 04:58:02.357949 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 18 04:58:02.357949 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 18 04:58:02.357949 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 18 04:58:02.358832 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 18 04:58:02.358832 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 18 04:58:02.358832 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 18 04:58:02.358832 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 18 04:58:02.358832 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:58:02.358832 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:58:02.358832 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:58:02.358832 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:58:02.358832 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:58:02.358832 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:58:02.358832 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:58:02.358832 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jun 18 04:58:02.358832 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:58:02.358832 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:58:02.358832 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:58:02.358832 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:58:02.359578 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:58:02.359578 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:58:02.359578 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:58:02.359578 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:58:02.359578 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 18 04:58:02.359578 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 18 04:58:02.359578 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 18 04:58:02.359578 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 18 04:58:02.359578 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 18 04:58:02.359578 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 18 04:58:02.359578 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 18 04:58:02.359578 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 04:58:02.359578 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 61/116] 10.222.0.23 () {56 vars in 1096 bytes} [Sat Jun 18 04:58:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:58:02.360843 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1376f0a-52cd-46a0-b6c9-d387ab77cb75 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:02.360843 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1376f0a-52cd-46a0-b6c9-d387ab77cb75 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:02.360843 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1376f0a-52cd-46a0-b6c9-d387ab77cb75 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:02.362500 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 62/117] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 04:58:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 04:58:02.363025 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1376f0a-52cd-46a0-b6c9-d387ab77cb75 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:02.366677 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:02.412194 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 56/118] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Jun 18 04:58:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 18 04:58:02.417509 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd44ee4b-cd6d-426e-a57f-e9feb9e6eae4 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:02.469214 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1376f0a-52cd-46a0-b6c9-d387ab77cb75 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:02.502606 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dec9d58-545e-4da1-8e37-0421f45b55c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:02.503262 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dec9d58-545e-4da1-8e37-0421f45b55c7 None admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:02.503548 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dec9d58-545e-4da1-8e37-0421f45b55c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:02.503832 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dec9d58-545e-4da1-8e37-0421f45b55c7 None admin] PATH_INFO: `/v3/projects` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:02.504539 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dec9d58-545e-4da1-8e37-0421f45b55c7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:02.506049 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dec9d58-545e-4da1-8e37-0421f45b55c7 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:02.513232 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd44ee4b-cd6d-426e-a57f-e9feb9e6eae4 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:02.546705 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30c8f04-c0b0-4219-88e1-4b0c15d713a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:02.547616 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30c8f04-c0b0-4219-88e1-4b0c15d713a8 None admin] REQUEST_METHOD: `PUT` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:02.547899 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30c8f04-c0b0-4219-88e1-4b0c15d713a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:02.548175 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30c8f04-c0b0-4219-88e1-4b0c15d713a8 None admin] PATH_INFO: `/v3/projects/6706fdb556af472c8319dd2a13abe441/users/21ca63be55244b159bcc68b518233c91/roles/cbf5c4c26898413d93e73b46b41046a3` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:02.550156 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30c8f04-c0b0-4219-88e1-4b0c15d713a8 None admin] RBAC: Authorizing `identity:create_grant(project_id=6706fdb556af472c8319dd2a13abe441, user_id=21ca63be55244b159bcc68b518233c91, role_id=cbf5c4c26898413d93e73b46b41046a3)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:02.551582 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30c8f04-c0b0-4219-88e1-4b0c15d713a8 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:02.603373 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 63/119] 10.222.0.23 () {60 vars in 1141 bytes} [Sat Jun 18 04:58:02 2022] POST /identity/v3/projects => generated 302 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 04:58:02.604209 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dec9d58-545e-4da1-8e37-0421f45b55c7 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:02.628883 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 57/120] 10.222.0.23 () {58 vars in 1439 bytes} [Sat Jun 18 04:58:02 2022] PUT /identity/v3/projects/6706fdb556af472c8319dd2a13abe441/users/21ca63be55244b159bcc68b518233c91/roles/cbf5c4c26898413d93e73b46b41046a3 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 04:58:02.630060 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30c8f04-c0b0-4219-88e1-4b0c15d713a8 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:02.635216 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:02.672624 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:02.751608 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dec9d58-545e-4da1-8e37-0421f45b55c7 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:02.763799 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30c8f04-c0b0-4219-88e1-4b0c15d713a8 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:02.806406 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de119061-ca10-4d5f-a046-b2a73d7ec81e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:02.807344 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de119061-ca10-4d5f-a046-b2a73d7ec81e None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:02.807635 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de119061-ca10-4d5f-a046-b2a73d7ec81e None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:02.808023 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de119061-ca10-4d5f-a046-b2a73d7ec81e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:02.808960 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d3fa1d-4c99-4fb6-b0f0-4b78f56aa888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:02.809920 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d3fa1d-4c99-4fb6-b0f0-4b78f56aa888 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:02.810423 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d3fa1d-4c99-4fb6-b0f0-4b78f56aa888 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:02.810922 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d3fa1d-4c99-4fb6-b0f0-4b78f56aa888 None admin] PATH_INFO: `/v3/roles` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:02.811936 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d3fa1d-4c99-4fb6-b0f0-4b78f56aa888 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:58:02.812955 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d3fa1d-4c99-4fb6-b0f0-4b78f56aa888 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:02.814644 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d3fa1d-4c99-4fb6-b0f0-4b78f56aa888 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:02.822239 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de119061-ca10-4d5f-a046-b2a73d7ec81e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 18 04:58:02.828614 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 58/121] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Jun 18 04:58:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:02.830324 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448e2a14-4a1e-4946-8321-d8800a0c6fc0 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:02.830803 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448e2a14-4a1e-4946-8321-d8800a0c6fc0 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:02.831281 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448e2a14-4a1e-4946-8321-d8800a0c6fc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:02.831537 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-de119061-ca10-4d5f-a046-b2a73d7ec81e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 18 04:58:02.831537 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:58:02.831537 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:58:02.831537 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:58:02.831537 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:58:02.831537 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:58:02.831537 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:58:02.831537 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:58:02.831537 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:58:02.831537 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:58:02.831537 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:58:02.831537 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:58:02.831537 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 18 04:58:02.831537 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 18 04:58:02.831537 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 18 04:58:02.831537 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 18 04:58:02.832345 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 18 04:58:02.832345 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 18 04:58:02.832345 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 18 04:58:02.832345 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 18 04:58:02.832345 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:58:02.832345 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:58:02.832345 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:58:02.832345 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:58:02.832345 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:58:02.832345 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:58:02.832345 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:58:02.832345 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jun 18 04:58:02.832345 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:58:02.832345 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:58:02.832345 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:58:02.832345 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:58:02.836481 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:58:02.836481 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:58:02.836481 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:58:02.836481 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:58:02.836481 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 18 04:58:02.836481 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 18 04:58:02.836481 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 18 04:58:02.836481 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 18 04:58:02.836481 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 18 04:58:02.836481 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 18 04:58:02.836481 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 18 04:58:02.836481 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 04:58:02.836481 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 64/122] 10.222.0.23 () {56 vars in 1114 bytes} [Sat Jun 18 04:58:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:58:02.836481 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de119061-ca10-4d5f-a046-b2a73d7ec81e None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:02.857739 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-448e2a14-4a1e-4946-8321-d8800a0c6fc0 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 04:58:02.859241 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-448e2a14-4a1e-4946-8321-d8800a0c6fc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 04:58:02.928207 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:02.937334 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 59/123] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 04:58:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 18 04:58:02.944440 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448e2a14-4a1e-4946-8321-d8800a0c6fc0 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:03.002037 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de119061-ca10-4d5f-a046-b2a73d7ec81e None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:03.019692 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:03.044940 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8deaefb-7f94-4c1f-a9bd-f656bb755935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:03.046000 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8deaefb-7f94-4c1f-a9bd-f656bb755935 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:03.046392 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8deaefb-7f94-4c1f-a9bd-f656bb755935 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:03.046778 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8deaefb-7f94-4c1f-a9bd-f656bb755935 None admin] PATH_INFO: `/v3/projects/6706fdb556af472c8319dd2a13abe441` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:03.048580 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8deaefb-7f94-4c1f-a9bd-f656bb755935 None admin] RBAC: Authorizing `identity:get_project(project_id=6706fdb556af472c8319dd2a13abe441)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:03.050591 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8deaefb-7f94-4c1f-a9bd-f656bb755935 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:03.054595 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 65/124] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Jun 18 04:58:02 2022] GET /identity/v3/projects/6706fdb556af472c8319dd2a13abe441 => generated 298 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:03.058019 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8deaefb-7f94-4c1f-a9bd-f656bb755935 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:03.087583 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448e2a14-4a1e-4946-8321-d8800a0c6fc0 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:03.132539 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36e3b64-bd7d-49bc-b775-f2134a3ad656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:03.134171 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36e3b64-bd7d-49bc-b775-f2134a3ad656 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:03.134608 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36e3b64-bd7d-49bc-b775-f2134a3ad656 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:03.135356 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36e3b64-bd7d-49bc-b775-f2134a3ad656 None admin] PATH_INFO: `/v3/users/5d3e83f3ca7347dea1350fa29bac5d9d` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:03.136931 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36e3b64-bd7d-49bc-b775-f2134a3ad656 None admin] RBAC: Authorizing `identity:get_user(user_id=5d3e83f3ca7347dea1350fa29bac5d9d)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:03.139242 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36e3b64-bd7d-49bc-b775-f2134a3ad656 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:03.141975 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 60/125] 10.222.0.23 () {56 vars in 1177 bytes} [Sat Jun 18 04:58:02 2022] GET /identity/v3/users/5d3e83f3ca7347dea1350fa29bac5d9d => generated 247 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:03.143211 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36e3b64-bd7d-49bc-b775-f2134a3ad656 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:03.196869 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8deaefb-7f94-4c1f-a9bd-f656bb755935 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:03.224220 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:03.242424 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2f0ef4-83da-46a5-b4cd-4fcb52121613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:03.247364 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2f0ef4-83da-46a5-b4cd-4fcb52121613 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:03.247364 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2f0ef4-83da-46a5-b4cd-4fcb52121613 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:03.247364 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2f0ef4-83da-46a5-b4cd-4fcb52121613 None admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:03.247364 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2f0ef4-83da-46a5-b4cd-4fcb52121613 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:58:03.247364 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2f0ef4-83da-46a5-b4cd-4fcb52121613 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:03.248482 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2f0ef4-83da-46a5-b4cd-4fcb52121613 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:03.260307 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 66/126] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Jun 18 04:58:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:03.265265 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2f0ef4-83da-46a5-b4cd-4fcb52121613 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:03.273633 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36e3b64-bd7d-49bc-b775-f2134a3ad656 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:03.313150 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee217e61-a732-4d12-aaec-bc8ddfc86727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:03.314127 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee217e61-a732-4d12-aaec-bc8ddfc86727 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:03.314514 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee217e61-a732-4d12-aaec-bc8ddfc86727 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:03.314888 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee217e61-a732-4d12-aaec-bc8ddfc86727 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:03.326882 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee217e61-a732-4d12-aaec-bc8ddfc86727 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 18 04:58:03.329779 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:03.335497 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application [None req-ee217e61-a732-4d12-aaec-bc8ddfc86727 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 18 04:58:03.335497 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:58:03.335497 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:58:03.335497 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:58:03.335497 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:58:03.335497 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:58:03.335497 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:58:03.335497 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:58:03.335497 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:58:03.335497 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:58:03.335497 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:58:03.335497 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:58:03.335497 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 18 04:58:03.335497 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 18 04:58:03.335497 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 18 04:58:03.335497 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 18 04:58:03.336667 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 18 04:58:03.336667 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 18 04:58:03.336667 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 18 04:58:03.336667 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 18 04:58:03.336667 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:58:03.336667 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:58:03.336667 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:58:03.336667 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:58:03.336667 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:58:03.336667 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:58:03.336667 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:58:03.336667 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application with Lock( Jun 18 04:58:03.336667 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:58:03.336667 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:58:03.336667 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:58:03.336667 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:58:03.337688 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:58:03.337688 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:58:03.337688 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:58:03.337688 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:58:03.337688 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 18 04:58:03.337688 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 18 04:58:03.337688 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 18 04:58:03.337688 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 18 04:58:03.337688 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 18 04:58:03.337688 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 18 04:58:03.337688 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 18 04:58:03.337688 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 04:58:03.339432 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 61/127] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Jun 18 04:58:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:58:03.345053 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee217e61-a732-4d12-aaec-bc8ddfc86727 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:03.403976 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2f0ef4-83da-46a5-b4cd-4fcb52121613 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:03.453257 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89671210-f5d6-4d0a-b2d6-2b5d0052c26b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:03.454531 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89671210-f5d6-4d0a-b2d6-2b5d0052c26b None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:03.454993 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89671210-f5d6-4d0a-b2d6-2b5d0052c26b None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:03.455433 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89671210-f5d6-4d0a-b2d6-2b5d0052c26b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:03.456416 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89671210-f5d6-4d0a-b2d6-2b5d0052c26b None admin] RBAC: Adding query filter params (role.id=80cac029fc6540658713c822fdd75949, scope.project.id=6706fdb556af472c8319dd2a13abe441, user.id=2b6fc67a02584c8cac37b9b3f2205270) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:58:03.457444 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89671210-f5d6-4d0a-b2d6-2b5d0052c26b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:03.459602 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89671210-f5d6-4d0a-b2d6-2b5d0052c26b None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:03.468687 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 67/128] 10.222.0.23 () {56 vars in 1374 bytes} [Sat Jun 18 04:58:03 2022] GET /identity/v3/role_assignments?user.id=2b6fc67a02584c8cac37b9b3f2205270&scope.project.id=6706fdb556af472c8319dd2a13abe441&role.id=80cac029fc6540658713c822fdd75949 => generated 259 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:03.469702 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89671210-f5d6-4d0a-b2d6-2b5d0052c26b None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:03.474863 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee217e61-a732-4d12-aaec-bc8ddfc86727 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:03.536618 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:03.558501 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05368765-0283-48f9-b4f6-a78672a4c885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:03.559555 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05368765-0283-48f9-b4f6-a78672a4c885 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:03.559923 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05368765-0283-48f9-b4f6-a78672a4c885 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:03.560305 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05368765-0283-48f9-b4f6-a78672a4c885 None admin] PATH_INFO: `/v3/projects/6706fdb556af472c8319dd2a13abe441` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:03.561803 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05368765-0283-48f9-b4f6-a78672a4c885 None admin] RBAC: Authorizing `identity:get_project(project_id=6706fdb556af472c8319dd2a13abe441)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:03.563360 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05368765-0283-48f9-b4f6-a78672a4c885 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:03.565356 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 62/129] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Jun 18 04:58:03 2022] GET /identity/v3/projects/6706fdb556af472c8319dd2a13abe441 => generated 298 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:03.574366 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05368765-0283-48f9-b4f6-a78672a4c885 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:03.629991 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89671210-f5d6-4d0a-b2d6-2b5d0052c26b None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:03.641073 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:03.678789 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd23847c-4a8d-4b0f-a2b3-19b5015f077c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:03.680163 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd23847c-4a8d-4b0f-a2b3-19b5015f077c None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:03.680163 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd23847c-4a8d-4b0f-a2b3-19b5015f077c None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:03.680377 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd23847c-4a8d-4b0f-a2b3-19b5015f077c None admin] PATH_INFO: `/v3/users/21ca63be55244b159bcc68b518233c91` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:03.681745 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd23847c-4a8d-4b0f-a2b3-19b5015f077c None admin] RBAC: Authorizing `identity:get_user(user_id=21ca63be55244b159bcc68b518233c91)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:03.683247 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd23847c-4a8d-4b0f-a2b3-19b5015f077c None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:03.685123 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 68/130] 10.222.0.23 () {56 vars in 1177 bytes} [Sat Jun 18 04:58:03 2022] GET /identity/v3/users/21ca63be55244b159bcc68b518233c91 => generated 275 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:03.691494 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd23847c-4a8d-4b0f-a2b3-19b5015f077c None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:03.744435 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05368765-0283-48f9-b4f6-a78672a4c885 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:03.790648 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb57049-4bf2-47ca-b95c-a36d271747f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:03.791236 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb57049-4bf2-47ca-b95c-a36d271747f6 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:03.791625 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb57049-4bf2-47ca-b95c-a36d271747f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:03.792007 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb57049-4bf2-47ca-b95c-a36d271747f6 None admin] PATH_INFO: `/v3/roles` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:03.792874 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb57049-4bf2-47ca-b95c-a36d271747f6 None admin] RBAC: Adding query filter params (name=member) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:58:03.793291 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb57049-4bf2-47ca-b95c-a36d271747f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:03.795165 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb57049-4bf2-47ca-b95c-a36d271747f6 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:03.810655 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 63/131] 10.222.0.23 () {56 vars in 1101 bytes} [Sat Jun 18 04:58:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:03.811406 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb57049-4bf2-47ca-b95c-a36d271747f6 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:03.811528 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd23847c-4a8d-4b0f-a2b3-19b5015f077c None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:03.878240 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccd7cd6-68f6-41fc-9781-48deeedb6124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:03.879153 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccd7cd6-68f6-41fc-9781-48deeedb6124 None admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:03.879445 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccd7cd6-68f6-41fc-9781-48deeedb6124 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:03.879731 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccd7cd6-68f6-41fc-9781-48deeedb6124 None admin] PATH_INFO: `/v3/projects/6706fdb556af472c8319dd2a13abe441/users/5d3e83f3ca7347dea1350fa29bac5d9d/roles/21c65f2c51bd4dd283d13507d2714467` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:03.883480 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccd7cd6-68f6-41fc-9781-48deeedb6124 None admin] RBAC: Authorizing `identity:create_grant(project_id=6706fdb556af472c8319dd2a13abe441, user_id=5d3e83f3ca7347dea1350fa29bac5d9d, role_id=21c65f2c51bd4dd283d13507d2714467)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:03.884954 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccd7cd6-68f6-41fc-9781-48deeedb6124 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:03.928685 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:03.949422 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb57049-4bf2-47ca-b95c-a36d271747f6 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:03.985154 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 69/132] 10.222.0.23 () {58 vars in 1439 bytes} [Sat Jun 18 04:58:03 2022] PUT /identity/v3/projects/6706fdb556af472c8319dd2a13abe441/users/5d3e83f3ca7347dea1350fa29bac5d9d/roles/21c65f2c51bd4dd283d13507d2714467 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 04:58:03.986012 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccd7cd6-68f6-41fc-9781-48deeedb6124 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:03.998019 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af27bb5-3617-45a0-b33c-a8144d69453f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:03.999127 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af27bb5-3617-45a0-b33c-a8144d69453f None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:03.999585 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af27bb5-3617-45a0-b33c-a8144d69453f None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:04.000039 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af27bb5-3617-45a0-b33c-a8144d69453f None admin] PATH_INFO: `/v3/projects/6706fdb556af472c8319dd2a13abe441` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:04.006708 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af27bb5-3617-45a0-b33c-a8144d69453f None admin] RBAC: Authorizing `identity:get_project(project_id=6706fdb556af472c8319dd2a13abe441)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:04.008606 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af27bb5-3617-45a0-b33c-a8144d69453f None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:04.011447 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 64/133] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Jun 18 04:58:03 2022] GET /identity/v3/projects/6706fdb556af472c8319dd2a13abe441 => generated 298 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:04.017080 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af27bb5-3617-45a0-b33c-a8144d69453f None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:04.034070 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:04.107432 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccd7cd6-68f6-41fc-9781-48deeedb6124 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:04.152486 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8a74e2-451b-41e2-9347-2ec01f4d4db3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:04.153408 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8a74e2-451b-41e2-9347-2ec01f4d4db3 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:04.153698 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8a74e2-451b-41e2-9347-2ec01f4d4db3 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:04.153985 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8a74e2-451b-41e2-9347-2ec01f4d4db3 None admin] PATH_INFO: `/v3/users/21ca63be55244b159bcc68b518233c91` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:04.155379 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8a74e2-451b-41e2-9347-2ec01f4d4db3 None admin] RBAC: Authorizing `identity:get_user(user_id=21ca63be55244b159bcc68b518233c91)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:04.156848 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8a74e2-451b-41e2-9347-2ec01f4d4db3 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:04.158541 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 70/134] 10.222.0.23 () {56 vars in 1177 bytes} [Sat Jun 18 04:58:03 2022] GET /identity/v3/users/21ca63be55244b159bcc68b518233c91 => generated 275 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:04.166691 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8a74e2-451b-41e2-9347-2ec01f4d4db3 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:04.215678 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af27bb5-3617-45a0-b33c-a8144d69453f None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:04.272792 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:04.311967 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8a74e2-451b-41e2-9347-2ec01f4d4db3 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:04.314145 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c354a96f-49b5-4ebc-8905-e41edd43eb01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:04.314640 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:04.315244 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c354a96f-49b5-4ebc-8905-e41edd43eb01 None admin] REQUEST_METHOD: `PUT` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:04.315435 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c354a96f-49b5-4ebc-8905-e41edd43eb01 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:04.315685 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c354a96f-49b5-4ebc-8905-e41edd43eb01 None admin] PATH_INFO: `/v3/projects/6706fdb556af472c8319dd2a13abe441/users/21ca63be55244b159bcc68b518233c91/roles/0089a3aad83c4ae08d8e9d864957285b` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:04.322932 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c354a96f-49b5-4ebc-8905-e41edd43eb01 None admin] RBAC: Authorizing `identity:create_grant(project_id=6706fdb556af472c8319dd2a13abe441, user_id=21ca63be55244b159bcc68b518233c91, role_id=0089a3aad83c4ae08d8e9d864957285b)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:04.324429 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c354a96f-49b5-4ebc-8905-e41edd43eb01 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:04.353112 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449b9a44-d31b-4c56-94e6-4d6c501a34af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:04.354029 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449b9a44-d31b-4c56-94e6-4d6c501a34af None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:04.354320 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449b9a44-d31b-4c56-94e6-4d6c501a34af None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:04.354664 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449b9a44-d31b-4c56-94e6-4d6c501a34af None admin] PATH_INFO: `/v3/projects/fa4a0c00884f4f97b9e8fc0496982551` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:04.355883 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449b9a44-d31b-4c56-94e6-4d6c501a34af None admin] RBAC: Authorizing `identity:get_project(project_id=fa4a0c00884f4f97b9e8fc0496982551)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:04.357347 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449b9a44-d31b-4c56-94e6-4d6c501a34af None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:04.359460 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 71/135] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Jun 18 04:58:04 2022] GET /identity/v3/projects/fa4a0c00884f4f97b9e8fc0496982551 => generated 312 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:04.365485 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449b9a44-d31b-4c56-94e6-4d6c501a34af None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:04.439731 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 65/136] 10.222.0.23 () {58 vars in 1439 bytes} [Sat Jun 18 04:58:04 2022] PUT /identity/v3/projects/6706fdb556af472c8319dd2a13abe441/users/21ca63be55244b159bcc68b518233c91/roles/0089a3aad83c4ae08d8e9d864957285b => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 04:58:04.479456 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449b9a44-d31b-4c56-94e6-4d6c501a34af None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:04.517737 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f40927d-bc21-4dc5-80b7-f1535bb3b9ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:04.518643 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f40927d-bc21-4dc5-80b7-f1535bb3b9ce None admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:04.518926 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f40927d-bc21-4dc5-80b7-f1535bb3b9ce None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:04.519205 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f40927d-bc21-4dc5-80b7-f1535bb3b9ce None admin] PATH_INFO: `/v3/projects/fa4a0c00884f4f97b9e8fc0496982551/users/21ca63be55244b159bcc68b518233c91/roles/cbf5c4c26898413d93e73b46b41046a3` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:04.521479 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f40927d-bc21-4dc5-80b7-f1535bb3b9ce None admin] RBAC: Authorizing `identity:create_grant(project_id=fa4a0c00884f4f97b9e8fc0496982551, user_id=21ca63be55244b159bcc68b518233c91, role_id=cbf5c4c26898413d93e73b46b41046a3)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:04.523620 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f40927d-bc21-4dc5-80b7-f1535bb3b9ce None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:04.607113 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 72/137] 10.222.0.23 () {58 vars in 1439 bytes} [Sat Jun 18 04:58:04 2022] PUT /identity/v3/projects/fa4a0c00884f4f97b9e8fc0496982551/users/21ca63be55244b159bcc68b518233c91/roles/cbf5c4c26898413d93e73b46b41046a3 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 18 04:58:05.152362 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c2463b-5711-45d4-8b43-948f76fcded4 None None] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:05.152890 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c2463b-5711-45d4-8b43-948f76fcded4 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:05.152978 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c2463b-5711-45d4-8b43-948f76fcded4 None None] PATH_INFO: `/` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:05.154481 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 66/138] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 04:58:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 04:58:05.159571 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f85b7b-4603-40c1-bf84-0122c4f4df14 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:05.159959 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f85b7b-4603-40c1-bf84-0122c4f4df14 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:05.160130 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f85b7b-4603-40c1-bf84-0122c4f4df14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:05.178445 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-d6f85b7b-4603-40c1-bf84-0122c4f4df14 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 04:58:05.179826 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-d6f85b7b-4603-40c1-bf84-0122c4f4df14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 04:58:05.207679 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:05.253137 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 73/139] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 04:58:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 18 04:58:05.278676 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c2463b-5711-45d4-8b43-948f76fcded4 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:05.306062 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a62144-d87b-4178-9241-7b622ecbc612 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:05.306390 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a62144-d87b-4178-9241-7b622ecbc612 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:05.306652 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a62144-d87b-4178-9241-7b622ecbc612 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:05.308258 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 74/140] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 04:58:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 04:58:05.321419 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a97a54-a5fe-46af-884d-a19b31b37c20 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:05.321419 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a97a54-a5fe-46af-884d-a19b31b37c20 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:05.321419 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a97a54-a5fe-46af-884d-a19b31b37c20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:05.345796 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-94a97a54-a5fe-46af-884d-a19b31b37c20 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 04:58:05.345796 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-94a97a54-a5fe-46af-884d-a19b31b37c20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 04:58:05.439054 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c2463b-5711-45d4-8b43-948f76fcded4 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:05.456575 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 75/141] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 04:58:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 18 04:58:05.490156 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff006a1e-66ea-4e1c-a29f-46cc659b1b0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:05.492244 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff006a1e-66ea-4e1c-a29f-46cc659b1b0c None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:05.492532 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff006a1e-66ea-4e1c-a29f-46cc659b1b0c None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:05.492812 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff006a1e-66ea-4e1c-a29f-46cc659b1b0c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:05.494035 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff006a1e-66ea-4e1c-a29f-46cc659b1b0c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:05.495599 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff006a1e-66ea-4e1c-a29f-46cc659b1b0c None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:05.501515 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 67/142] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Jun 18 04:58:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:05.503422 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a97a54-a5fe-46af-884d-a19b31b37c20 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:05.603404 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff006a1e-66ea-4e1c-a29f-46cc659b1b0c None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:05.633411 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:05.640803 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a97a54-a5fe-46af-884d-a19b31b37c20 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:05.681592 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:05.682581 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a3a947-746f-4511-bb67-361a9ddb1ab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:05.683264 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a3a947-746f-4511-bb67-361a9ddb1ab0 None admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:05.683547 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a3a947-746f-4511-bb67-361a9ddb1ab0 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:05.683822 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a3a947-746f-4511-bb67-361a9ddb1ab0 None admin] PATH_INFO: `/v3/users` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:05.689476 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a3a947-746f-4511-bb67-361a9ddb1ab0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:05.689476 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a3a947-746f-4511-bb67-361a9ddb1ab0 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:05.740144 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff006a1e-66ea-4e1c-a29f-46cc659b1b0c None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:05.775811 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bd329f-7496-4935-9a0a-fc7eba0e7210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:05.776990 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bd329f-7496-4935-9a0a-fc7eba0e7210 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:05.777513 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bd329f-7496-4935-9a0a-fc7eba0e7210 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:05.778002 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bd329f-7496-4935-9a0a-fc7eba0e7210 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:05.789783 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bd329f-7496-4935-9a0a-fc7eba0e7210 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 18 04:58:05.797756 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application [None req-a4bd329f-7496-4935-9a0a-fc7eba0e7210 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 18 04:58:05.797756 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:58:05.797756 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:58:05.797756 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:58:05.797756 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:58:05.797756 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:58:05.797756 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:58:05.797756 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:58:05.797756 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:58:05.797756 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:58:05.797756 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:58:05.797756 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:58:05.797756 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 18 04:58:05.797756 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 18 04:58:05.797756 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 18 04:58:05.797756 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 18 04:58:05.798624 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 18 04:58:05.798624 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 18 04:58:05.798624 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 18 04:58:05.798624 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 18 04:58:05.798624 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:58:05.798624 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:58:05.798624 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:58:05.798624 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:58:05.798624 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:58:05.798624 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:58:05.798624 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:58:05.798624 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application with Lock( Jun 18 04:58:05.798624 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:58:05.798624 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:58:05.798624 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:58:05.798624 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:58:05.799404 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:58:05.799404 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:58:05.799404 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:58:05.799404 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:58:05.799404 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 18 04:58:05.799404 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 18 04:58:05.799404 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 18 04:58:05.799404 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 18 04:58:05.799404 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 18 04:58:05.799404 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 18 04:58:05.799404 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 18 04:58:05.799404 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 04:58:05.802084 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 68/143] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Jun 18 04:58:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:58:05.809789 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bd329f-7496-4935-9a0a-fc7eba0e7210 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:05.816059 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 76/144] 10.222.0.23 () {60 vars in 1132 bytes} [Sat Jun 18 04:58:05 2022] POST /identity/v3/users => generated 283 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 04:58:05.927001 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bd329f-7496-4935-9a0a-fc7eba0e7210 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:05.945888 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:05.971792 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a37a44-3e4b-4b26-baeb-ff7d2ee8a31f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:05.972816 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a37a44-3e4b-4b26-baeb-ff7d2ee8a31f None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:05.973283 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a37a44-3e4b-4b26-baeb-ff7d2ee8a31f None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:05.973774 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a37a44-3e4b-4b26-baeb-ff7d2ee8a31f None admin] PATH_INFO: `/v3/roles` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:05.974763 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a37a44-3e4b-4b26-baeb-ff7d2ee8a31f None admin] RBAC: Adding query filter params (name=member) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:58:05.975297 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a37a44-3e4b-4b26-baeb-ff7d2ee8a31f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:05.977057 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a37a44-3e4b-4b26-baeb-ff7d2ee8a31f None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:05.993425 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 69/145] 10.222.0.23 () {56 vars in 1101 bytes} [Sat Jun 18 04:58:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 18 04:58:05.995574 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a3a947-746f-4511-bb67-361a9ddb1ab0 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:06.118904 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a3a947-746f-4511-bb67-361a9ddb1ab0 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:06.155812 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac3f18e-0076-48f5-9a58-331dc51f3607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:06.156691 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac3f18e-0076-48f5-9a58-331dc51f3607 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:06.156974 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac3f18e-0076-48f5-9a58-331dc51f3607 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:06.157247 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac3f18e-0076-48f5-9a58-331dc51f3607 None admin] PATH_INFO: `/v3/users/21ca63be55244b159bcc68b518233c91` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:06.158471 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac3f18e-0076-48f5-9a58-331dc51f3607 None admin] RBAC: Authorizing `identity:get_user(user_id=21ca63be55244b159bcc68b518233c91)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:06.159935 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac3f18e-0076-48f5-9a58-331dc51f3607 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:06.163631 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 77/146] 10.222.0.23 () {56 vars in 1177 bytes} [Sat Jun 18 04:58:05 2022] GET /identity/v3/users/21ca63be55244b159bcc68b518233c91 => generated 275 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:06.168449 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a37a44-3e4b-4b26-baeb-ff7d2ee8a31f None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:06.193162 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c25b036-f266-4c40-be75-b46a19d72103 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:06.193519 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c25b036-f266-4c40-be75-b46a19d72103 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:06.194673 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c25b036-f266-4c40-be75-b46a19d72103 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:06.196548 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 78/147] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 04:58:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 18 04:58:06.201497 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaa2078-9411-495a-b969-afed76a26037 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:06.201743 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaa2078-9411-495a-b969-afed76a26037 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:06.202196 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaa2078-9411-495a-b969-afed76a26037 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:06.225163 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-0aaa2078-9411-495a-b969-afed76a26037 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 04:58:06.227309 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-0aaa2078-9411-495a-b969-afed76a26037 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 04:58:06.249193 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:06.293521 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a37a44-3e4b-4b26-baeb-ff7d2ee8a31f None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:06.332296 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfb2df2-4a9a-4167-9d5d-8c01e2672701 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:06.333311 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfb2df2-4a9a-4167-9d5d-8c01e2672701 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:06.333821 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfb2df2-4a9a-4167-9d5d-8c01e2672701 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:06.334885 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfb2df2-4a9a-4167-9d5d-8c01e2672701 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:06.347408 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 79/148] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 04:58:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 18 04:58:06.379151 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aaa2078-9411-495a-b969-afed76a26037 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:06.397835 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:06.420921 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfb2df2-4a9a-4167-9d5d-8c01e2672701 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:06.422820 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfb2df2-4a9a-4167-9d5d-8c01e2672701 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:06.526965 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 70/149] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Jun 18 04:58:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 18 04:58:06.532502 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfb2df2-4a9a-4167-9d5d-8c01e2672701 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:06.566084 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aaa2078-9411-495a-b969-afed76a26037 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:06.613271 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:06.638107 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974605f3-9a7a-422c-8181-c9246459e723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:06.639280 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974605f3-9a7a-422c-8181-c9246459e723 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:06.639773 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974605f3-9a7a-422c-8181-c9246459e723 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:06.640276 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974605f3-9a7a-422c-8181-c9246459e723 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:06.646596 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfb2df2-4a9a-4167-9d5d-8c01e2672701 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:06.655630 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974605f3-9a7a-422c-8181-c9246459e723 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 18 04:58:06.666533 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-974605f3-9a7a-422c-8181-c9246459e723 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 18 04:58:06.666533 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:58:06.666533 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:58:06.666533 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:58:06.666533 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:58:06.666533 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:58:06.666533 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:58:06.666533 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:58:06.666533 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:58:06.666533 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:58:06.666533 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:58:06.666533 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:58:06.666533 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 18 04:58:06.666533 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 18 04:58:06.666533 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 18 04:58:06.666533 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 18 04:58:06.667641 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 18 04:58:06.667641 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 18 04:58:06.667641 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 18 04:58:06.667641 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 18 04:58:06.667641 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:58:06.667641 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:58:06.667641 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:58:06.667641 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:58:06.667641 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:58:06.667641 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:58:06.667641 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:58:06.667641 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jun 18 04:58:06.667641 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:58:06.667641 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:58:06.667641 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:58:06.667641 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:58:06.668609 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:58:06.668609 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:58:06.668609 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:58:06.668609 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:58:06.668609 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 18 04:58:06.668609 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 18 04:58:06.668609 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 18 04:58:06.668609 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 18 04:58:06.668609 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 18 04:58:06.668609 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 18 04:58:06.668609 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 18 04:58:06.668609 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 04:58:06.670592 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 80/150] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Jun 18 04:58:06 2022] GET /identity/v3/roles/reader => generated 84 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 18 04:58:06.686505 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974605f3-9a7a-422c-8181-c9246459e723 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:06.709515 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:06.730280 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fa74fc-6596-4563-8ed7-d83a32de327d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:06.731492 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fa74fc-6596-4563-8ed7-d83a32de327d None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:06.732036 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fa74fc-6596-4563-8ed7-d83a32de327d None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:06.732525 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fa74fc-6596-4563-8ed7-d83a32de327d None admin] PATH_INFO: `/v3/projects/6706fdb556af472c8319dd2a13abe441` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:06.734116 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fa74fc-6596-4563-8ed7-d83a32de327d None admin] RBAC: Authorizing `identity:get_project(project_id=6706fdb556af472c8319dd2a13abe441)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:06.735860 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fa74fc-6596-4563-8ed7-d83a32de327d None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:06.745891 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 71/151] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Jun 18 04:58:06 2022] GET /identity/v3/projects/6706fdb556af472c8319dd2a13abe441 => generated 298 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:06.749404 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fa74fc-6596-4563-8ed7-d83a32de327d None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:06.900061 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974605f3-9a7a-422c-8181-c9246459e723 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:06.939381 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fa74fc-6596-4563-8ed7-d83a32de327d None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:06.947811 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63e0fb3-71de-4042-9942-21979a7248dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:06.948500 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63e0fb3-71de-4042-9942-21979a7248dc None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:06.948789 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63e0fb3-71de-4042-9942-21979a7248dc None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:06.949067 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63e0fb3-71de-4042-9942-21979a7248dc None admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:06.952554 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63e0fb3-71de-4042-9942-21979a7248dc None admin] RBAC: Adding query filter params (name=reader) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:58:06.952554 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63e0fb3-71de-4042-9942-21979a7248dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:06.952554 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63e0fb3-71de-4042-9942-21979a7248dc None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:06.969455 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 81/152] 10.222.0.23 () {56 vars in 1101 bytes} [Sat Jun 18 04:58:06 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:06.974295 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63e0fb3-71de-4042-9942-21979a7248dc None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:07.040131 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8d22a6-18d9-4e37-9a0d-f40ba28f3430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:07.041423 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8d22a6-18d9-4e37-9a0d-f40ba28f3430 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:07.041423 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8d22a6-18d9-4e37-9a0d-f40ba28f3430 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:07.041423 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8d22a6-18d9-4e37-9a0d-f40ba28f3430 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:07.042244 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8d22a6-18d9-4e37-9a0d-f40ba28f3430 None admin] RBAC: Adding query filter params (role.id=cbf5c4c26898413d93e73b46b41046a3, scope.project.id=6706fdb556af472c8319dd2a13abe441, user.id=21ca63be55244b159bcc68b518233c91) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:58:07.042571 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8d22a6-18d9-4e37-9a0d-f40ba28f3430 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:07.044256 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8d22a6-18d9-4e37-9a0d-f40ba28f3430 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:07.060884 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 72/153] 10.222.0.23 () {56 vars in 1374 bytes} [Sat Jun 18 04:58:06 2022] GET /identity/v3/role_assignments?user.id=21ca63be55244b159bcc68b518233c91&scope.project.id=6706fdb556af472c8319dd2a13abe441&role.id=cbf5c4c26898413d93e73b46b41046a3 => generated 608 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:07.140720 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:07.187164 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63e0fb3-71de-4042-9942-21979a7248dc None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:07.235802 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8846419d-567c-465c-8bac-846518a603c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:07.237073 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8846419d-567c-465c-8bac-846518a603c6 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:07.237784 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8846419d-567c-465c-8bac-846518a603c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:07.238276 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8846419d-567c-465c-8bac-846518a603c6 None admin] PATH_INFO: `/v3/users/2b6fc67a02584c8cac37b9b3f2205270` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:07.242432 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8846419d-567c-465c-8bac-846518a603c6 None admin] RBAC: Authorizing `identity:get_user(user_id=2b6fc67a02584c8cac37b9b3f2205270)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:07.243936 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8846419d-567c-465c-8bac-846518a603c6 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:07.245957 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 82/154] 10.222.0.23 () {56 vars in 1177 bytes} [Sat Jun 18 04:58:06 2022] GET /identity/v3/users/2b6fc67a02584c8cac37b9b3f2205270 => generated 289 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:07.251318 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8d22a6-18d9-4e37-9a0d-f40ba28f3430 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:07.255315 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:07.389642 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8d22a6-18d9-4e37-9a0d-f40ba28f3430 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:07.429698 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08391f76-5c94-4076-8afa-a1a9055c7df4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:07.430619 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08391f76-5c94-4076-8afa-a1a9055c7df4 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:07.430916 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08391f76-5c94-4076-8afa-a1a9055c7df4 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:07.431198 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08391f76-5c94-4076-8afa-a1a9055c7df4 None admin] PATH_INFO: `/v3/projects/6706fdb556af472c8319dd2a13abe441` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:07.432431 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08391f76-5c94-4076-8afa-a1a9055c7df4 None admin] RBAC: Authorizing `identity:get_project(project_id=6706fdb556af472c8319dd2a13abe441)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:07.433932 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08391f76-5c94-4076-8afa-a1a9055c7df4 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:07.435684 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 73/155] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Jun 18 04:58:07 2022] GET /identity/v3/projects/6706fdb556af472c8319dd2a13abe441 => generated 298 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:07.441551 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8846419d-567c-465c-8bac-846518a603c6 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:07.559835 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8846419d-567c-465c-8bac-846518a603c6 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:07.596591 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41782d2-c099-4593-8c02-27c60dbbe055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:07.597517 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41782d2-c099-4593-8c02-27c60dbbe055 None admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:07.597807 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41782d2-c099-4593-8c02-27c60dbbe055 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:07.598086 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41782d2-c099-4593-8c02-27c60dbbe055 None admin] PATH_INFO: `/v3/projects/6706fdb556af472c8319dd2a13abe441/users/2b6fc67a02584c8cac37b9b3f2205270/roles/80cac029fc6540658713c822fdd75949` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:07.601529 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41782d2-c099-4593-8c02-27c60dbbe055 None admin] RBAC: Authorizing `identity:create_grant(project_id=6706fdb556af472c8319dd2a13abe441, user_id=2b6fc67a02584c8cac37b9b3f2205270, role_id=80cac029fc6540658713c822fdd75949)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:07.603017 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41782d2-c099-4593-8c02-27c60dbbe055 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:07.606316 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:07.688810 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 83/156] 10.222.0.23 () {58 vars in 1439 bytes} [Sat Jun 18 04:58:07 2022] PUT /identity/v3/projects/6706fdb556af472c8319dd2a13abe441/users/2b6fc67a02584c8cac37b9b3f2205270/roles/80cac029fc6540658713c822fdd75949 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 04:58:07.830880 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f043a5b4-b6af-4035-88d4-4f3637d6a709 None None] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:07.831210 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f043a5b4-b6af-4035-88d4-4f3637d6a709 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:07.831382 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f043a5b4-b6af-4035-88d4-4f3637d6a709 None None] PATH_INFO: `/` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:07.832879 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 74/157] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 04:58:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 04:58:07.840258 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63134caa-45f0-4bb1-9943-d2109b966ef9 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:07.840567 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63134caa-45f0-4bb1-9943-d2109b966ef9 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:07.840887 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63134caa-45f0-4bb1-9943-d2109b966ef9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:07.863230 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-63134caa-45f0-4bb1-9943-d2109b966ef9 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 04:58:07.864743 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-63134caa-45f0-4bb1-9943-d2109b966ef9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 04:58:07.942997 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 84/158] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 04:58:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 18 04:58:07.982081 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f043a5b4-b6af-4035-88d4-4f3637d6a709 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:08.013571 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfcd87f-e0b1-4011-898a-3889fd336bea None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:08.013870 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfcd87f-e0b1-4011-898a-3889fd336bea None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:08.014143 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfcd87f-e0b1-4011-898a-3889fd336bea None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:08.015544 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 85/159] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 04:58:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 18 04:58:08.026522 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f56afde-d9ad-4953-8cc9-7fc93e6cf919 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:08.026802 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f56afde-d9ad-4953-8cc9-7fc93e6cf919 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:08.027078 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f56afde-d9ad-4953-8cc9-7fc93e6cf919 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:08.051065 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-4f56afde-d9ad-4953-8cc9-7fc93e6cf919 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 04:58:08.052270 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-4f56afde-d9ad-4953-8cc9-7fc93e6cf919 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 04:58:08.124730 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:08.144243 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 86/160] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 04:58:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 18 04:58:08.147094 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223f4a01-ca2b-493d-bf5a-5faa483d2923 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:08.147398 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223f4a01-ca2b-493d-bf5a-5faa483d2923 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:08.147864 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223f4a01-ca2b-493d-bf5a-5faa483d2923 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:08.149227 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 87/161] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 04:58:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 04:58:08.158392 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f043a5b4-b6af-4035-88d4-4f3637d6a709 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:08.159785 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026f3f24-ebb9-4008-b023-71fe7620eea2 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:08.160073 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026f3f24-ebb9-4008-b023-71fe7620eea2 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:08.160350 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026f3f24-ebb9-4008-b023-71fe7620eea2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:08.193531 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-026f3f24-ebb9-4008-b023-71fe7620eea2 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 04:58:08.194854 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-026f3f24-ebb9-4008-b023-71fe7620eea2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 04:58:08.232561 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57aea989-849c-4f1e-b8bc-d065c43a1f7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:08.233778 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aea989-849c-4f1e-b8bc-d065c43a1f7f None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:08.234262 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aea989-849c-4f1e-b8bc-d065c43a1f7f None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:08.234749 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aea989-849c-4f1e-b8bc-d065c43a1f7f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:08.268057 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57aea989-849c-4f1e-b8bc-d065c43a1f7f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 18 04:58:08.276772 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application [None req-57aea989-849c-4f1e-b8bc-d065c43a1f7f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 18 04:58:08.276772 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:58:08.276772 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:58:08.276772 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:58:08.276772 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:58:08.276772 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:58:08.276772 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:58:08.276772 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:58:08.276772 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:58:08.276772 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:58:08.276772 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:58:08.276772 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:58:08.276772 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 18 04:58:08.276772 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 18 04:58:08.276772 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 18 04:58:08.276772 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 18 04:58:08.277713 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 18 04:58:08.277713 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 18 04:58:08.277713 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 18 04:58:08.277713 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 18 04:58:08.277713 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:58:08.277713 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:58:08.277713 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:58:08.277713 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:58:08.277713 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:58:08.277713 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:58:08.277713 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:58:08.277713 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application with Lock( Jun 18 04:58:08.277713 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:58:08.277713 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:58:08.277713 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:58:08.277713 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:58:08.278481 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:58:08.278481 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:58:08.278481 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:58:08.278481 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:58:08.278481 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 18 04:58:08.278481 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 18 04:58:08.278481 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 18 04:58:08.278481 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 18 04:58:08.278481 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 18 04:58:08.278481 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 18 04:58:08.278481 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 18 04:58:08.278481 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 04:58:08.286146 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 75/162] 10.222.0.23 () {56 vars in 1114 bytes} [Sat Jun 18 04:58:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:58:08.286731 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:08.287659 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57aea989-849c-4f1e-b8bc-d065c43a1f7f None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:08.308039 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 88/163] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 04:58:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 18 04:58:08.309239 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026f3f24-ebb9-4008-b023-71fe7620eea2 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:08.500420 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026f3f24-ebb9-4008-b023-71fe7620eea2 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:08.537256 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57aea989-849c-4f1e-b8bc-d065c43a1f7f None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:08.571936 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8d74d1-4a85-4437-b589-833bbbc81bba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:08.572989 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8d74d1-4a85-4437-b589-833bbbc81bba None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:08.573516 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8d74d1-4a85-4437-b589-833bbbc81bba None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:08.573990 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8d74d1-4a85-4437-b589-833bbbc81bba None admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:08.574955 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8d74d1-4a85-4437-b589-833bbbc81bba None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:58:08.575476 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8d74d1-4a85-4437-b589-833bbbc81bba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:08.577091 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8d74d1-4a85-4437-b589-833bbbc81bba None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:08.592133 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 89/164] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Jun 18 04:58:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:08.593313 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8d74d1-4a85-4437-b589-833bbbc81bba None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:08.598402 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:08.611557 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc88d3e-4819-4b63-9beb-2a18de8f5459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:08.612899 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc88d3e-4819-4b63-9beb-2a18de8f5459 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:08.612899 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc88d3e-4819-4b63-9beb-2a18de8f5459 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:08.613056 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc88d3e-4819-4b63-9beb-2a18de8f5459 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:08.635883 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc88d3e-4819-4b63-9beb-2a18de8f5459 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 18 04:58:08.643579 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application [None req-4bc88d3e-4819-4b63-9beb-2a18de8f5459 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 18 04:58:08.643579 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:58:08.643579 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:58:08.643579 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:58:08.643579 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:58:08.643579 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:58:08.643579 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:58:08.643579 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:58:08.643579 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:58:08.643579 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:58:08.643579 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:58:08.643579 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:58:08.643579 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 18 04:58:08.643579 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 18 04:58:08.643579 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 18 04:58:08.643579 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 18 04:58:08.644444 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 18 04:58:08.644444 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 18 04:58:08.644444 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 18 04:58:08.644444 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 18 04:58:08.644444 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:58:08.644444 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:58:08.644444 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:58:08.644444 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:58:08.644444 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:58:08.644444 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:58:08.644444 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:58:08.644444 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application with Lock( Jun 18 04:58:08.644444 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:58:08.644444 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:58:08.644444 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:58:08.644444 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:58:08.645268 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:58:08.645268 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:58:08.645268 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:58:08.645268 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:58:08.645268 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 18 04:58:08.645268 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 18 04:58:08.645268 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 18 04:58:08.645268 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 18 04:58:08.645268 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 18 04:58:08.645268 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 18 04:58:08.645268 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 18 04:58:08.645268 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 04:58:08.645268 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 76/165] 10.222.0.23 () {56 vars in 1096 bytes} [Sat Jun 18 04:58:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 358 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:58:08.645943 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc88d3e-4819-4b63-9beb-2a18de8f5459 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:08.684654 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:08.797892 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8d74d1-4a85-4437-b589-833bbbc81bba None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:08.801212 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc88d3e-4819-4b63-9beb-2a18de8f5459 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:08.879240 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce458a56-bdb2-4450-8c15-42d6654167ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:08.880229 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce458a56-bdb2-4450-8c15-42d6654167ba None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:08.880521 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce458a56-bdb2-4450-8c15-42d6654167ba None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:08.880798 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce458a56-bdb2-4450-8c15-42d6654167ba None admin] PATH_INFO: `/v3/users/21ca63be55244b159bcc68b518233c91` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:08.882721 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce458a56-bdb2-4450-8c15-42d6654167ba None admin] RBAC: Authorizing `identity:get_user(user_id=21ca63be55244b159bcc68b518233c91)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:08.884290 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce458a56-bdb2-4450-8c15-42d6654167ba None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:08.886099 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 77/166] 10.222.0.23 () {56 vars in 1177 bytes} [Sat Jun 18 04:58:08 2022] GET /identity/v3/users/21ca63be55244b159bcc68b518233c91 => generated 275 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:08.886852 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce458a56-bdb2-4450-8c15-42d6654167ba None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:08.890179 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a6fa59-22e5-41ae-a05b-04cd84352e85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:08.891081 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a6fa59-22e5-41ae-a05b-04cd84352e85 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:08.891371 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a6fa59-22e5-41ae-a05b-04cd84352e85 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:08.891658 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a6fa59-22e5-41ae-a05b-04cd84352e85 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:08.918382 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a6fa59-22e5-41ae-a05b-04cd84352e85 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 18 04:58:08.926373 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-d5a6fa59-22e5-41ae-a05b-04cd84352e85 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 18 04:58:08.926373 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:58:08.926373 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:58:08.926373 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:58:08.926373 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:58:08.926373 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:58:08.926373 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:58:08.926373 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:58:08.926373 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:58:08.926373 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:58:08.926373 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:58:08.926373 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:58:08.926373 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 18 04:58:08.926373 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 18 04:58:08.926373 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 18 04:58:08.926373 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 18 04:58:08.927218 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 18 04:58:08.927218 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 18 04:58:08.927218 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 18 04:58:08.927218 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 18 04:58:08.927218 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:58:08.927218 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:58:08.927218 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:58:08.927218 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:58:08.927218 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:58:08.927218 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:58:08.927218 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:58:08.927218 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jun 18 04:58:08.927218 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:58:08.927218 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:58:08.927218 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:58:08.927218 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:58:08.928117 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:58:08.928117 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:58:08.928117 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:58:08.928117 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:58:08.928117 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 18 04:58:08.928117 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 18 04:58:08.928117 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 18 04:58:08.928117 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 18 04:58:08.928117 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 18 04:58:08.928117 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 18 04:58:08.928117 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 18 04:58:08.928117 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 04:58:08.932366 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 90/167] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Jun 18 04:58:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:58:08.933326 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a6fa59-22e5-41ae-a05b-04cd84352e85 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:09.002783 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:09.079248 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce458a56-bdb2-4450-8c15-42d6654167ba None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:09.138269 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a6fa59-22e5-41ae-a05b-04cd84352e85 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:09.149999 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d375666e-0c32-49bc-a478-7617f19d209d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:09.151004 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d375666e-0c32-49bc-a478-7617f19d209d None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:09.151517 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d375666e-0c32-49bc-a478-7617f19d209d None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:09.152047 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d375666e-0c32-49bc-a478-7617f19d209d None admin] PATH_INFO: `/v3/roles` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:09.152999 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d375666e-0c32-49bc-a478-7617f19d209d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:58:09.153546 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d375666e-0c32-49bc-a478-7617f19d209d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:09.155160 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d375666e-0c32-49bc-a478-7617f19d209d None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:09.156083 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:09.172886 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 78/168] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Jun 18 04:58:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:09.174060 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d375666e-0c32-49bc-a478-7617f19d209d None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:09.206114 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099ebde1-18a1-408c-9acf-016ab326381a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:09.206776 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099ebde1-18a1-408c-9acf-016ab326381a None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:09.207056 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099ebde1-18a1-408c-9acf-016ab326381a None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:09.207336 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099ebde1-18a1-408c-9acf-016ab326381a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:09.329803 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099ebde1-18a1-408c-9acf-016ab326381a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:09.331407 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099ebde1-18a1-408c-9acf-016ab326381a None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:09.379932 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d375666e-0c32-49bc-a478-7617f19d209d None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:09.432158 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:09.454988 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0591dccf-1ab9-49ca-a146-e012912ec419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:09.456022 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0591dccf-1ab9-49ca-a146-e012912ec419 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:09.456519 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0591dccf-1ab9-49ca-a146-e012912ec419 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:09.457033 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0591dccf-1ab9-49ca-a146-e012912ec419 None admin] PATH_INFO: `/v3/roles` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:09.458017 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0591dccf-1ab9-49ca-a146-e012912ec419 None admin] RBAC: Adding query filter params (name=member) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:58:09.458566 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0591dccf-1ab9-49ca-a146-e012912ec419 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:09.460221 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0591dccf-1ab9-49ca-a146-e012912ec419 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:09.477190 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 79/169] 10.222.0.23 () {56 vars in 1101 bytes} [Sat Jun 18 04:58:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:09.478352 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0591dccf-1ab9-49ca-a146-e012912ec419 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:09.543424 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 91/170] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Jun 18 04:58:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 610 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 18 04:58:09.544223 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099ebde1-18a1-408c-9acf-016ab326381a None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:09.565631 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:09.670815 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0591dccf-1ab9-49ca-a146-e012912ec419 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:09.755129 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db227b34-1524-489e-928c-4e950430d6f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:09.755922 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db227b34-1524-489e-928c-4e950430d6f6 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:09.756744 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db227b34-1524-489e-928c-4e950430d6f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:09.757045 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db227b34-1524-489e-928c-4e950430d6f6 None admin] PATH_INFO: `/v3/users/5d3e83f3ca7347dea1350fa29bac5d9d` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:09.762093 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db227b34-1524-489e-928c-4e950430d6f6 None admin] RBAC: Authorizing `identity:get_user(user_id=5d3e83f3ca7347dea1350fa29bac5d9d)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:09.763702 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db227b34-1524-489e-928c-4e950430d6f6 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:09.765483 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 80/171] 10.222.0.23 () {56 vars in 1177 bytes} [Sat Jun 18 04:58:09 2022] GET /identity/v3/users/5d3e83f3ca7347dea1350fa29bac5d9d => generated 247 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:09.766244 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db227b34-1524-489e-928c-4e950430d6f6 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:09.797124 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099ebde1-18a1-408c-9acf-016ab326381a None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:09.864654 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:09.878110 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d822ac-078c-4792-a438-f33a187b4932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:09.889500 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d822ac-078c-4792-a438-f33a187b4932 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:09.889500 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d822ac-078c-4792-a438-f33a187b4932 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:09.889500 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d822ac-078c-4792-a438-f33a187b4932 None admin] PATH_INFO: `/v3/users/21ca63be55244b159bcc68b518233c91` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:09.889500 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d822ac-078c-4792-a438-f33a187b4932 None admin] RBAC: Authorizing `identity:get_user(user_id=21ca63be55244b159bcc68b518233c91)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:09.889500 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d822ac-078c-4792-a438-f33a187b4932 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:09.889500 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 92/172] 10.222.0.23 () {56 vars in 1177 bytes} [Sat Jun 18 04:58:09 2022] GET /identity/v3/users/21ca63be55244b159bcc68b518233c91 => generated 275 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:09.889907 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d822ac-078c-4792-a438-f33a187b4932 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:09.927298 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db227b34-1524-489e-928c-4e950430d6f6 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:09.962543 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a483b93-8796-4c1a-bc03-1730b2f6e8c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:09.963415 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a483b93-8796-4c1a-bc03-1730b2f6e8c9 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:09.963697 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a483b93-8796-4c1a-bc03-1730b2f6e8c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:09.963967 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a483b93-8796-4c1a-bc03-1730b2f6e8c9 None admin] PATH_INFO: `/v3/projects/6706fdb556af472c8319dd2a13abe441` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:09.965164 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a483b93-8796-4c1a-bc03-1730b2f6e8c9 None admin] RBAC: Authorizing `identity:get_project(project_id=6706fdb556af472c8319dd2a13abe441)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:09.966665 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a483b93-8796-4c1a-bc03-1730b2f6e8c9 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:09.968370 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 81/173] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Jun 18 04:58:09 2022] GET /identity/v3/projects/6706fdb556af472c8319dd2a13abe441 => generated 298 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:09.969573 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed73a68-d379-4d56-8d09-50587207c5d0 None None] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:09.969854 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed73a68-d379-4d56-8d09-50587207c5d0 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:09.970131 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed73a68-d379-4d56-8d09-50587207c5d0 None None] PATH_INFO: `/` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:09.976433 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 82/174] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 04:58:09 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 04:58:09.977265 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed73a68-d379-4d56-8d09-50587207c5d0 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:10.078415 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:10.118151 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed73a68-d379-4d56-8d09-50587207c5d0 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:10.129004 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d822ac-078c-4792-a438-f33a187b4932 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:10.165775 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:10.183618 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645d084a-7e35-4a9f-b934-cab7b82557fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:10.184268 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645d084a-7e35-4a9f-b934-cab7b82557fa None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:10.184549 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645d084a-7e35-4a9f-b934-cab7b82557fa None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:10.184825 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645d084a-7e35-4a9f-b934-cab7b82557fa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:10.200045 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bc2b63-23b0-4a1b-b578-0ac9353cea08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:10.200699 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bc2b63-23b0-4a1b-b578-0ac9353cea08 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:10.200979 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bc2b63-23b0-4a1b-b578-0ac9353cea08 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:10.201248 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bc2b63-23b0-4a1b-b578-0ac9353cea08 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:10.287689 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645d084a-7e35-4a9f-b934-cab7b82557fa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:10.289621 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645d084a-7e35-4a9f-b934-cab7b82557fa None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:10.325550 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bc2b63-23b0-4a1b-b578-0ac9353cea08 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:10.327055 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bc2b63-23b0-4a1b-b578-0ac9353cea08 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:10.392305 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 83/175] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Jun 18 04:58:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 18 04:58:10.392905 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645d084a-7e35-4a9f-b934-cab7b82557fa None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:10.407681 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:10.421796 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:10.433165 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 93/176] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Jun 18 04:58:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 548 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 18 04:58:10.434787 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36e1a15-0be1-4ef0-8d95-bce5f307d3cf None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:10.435306 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36e1a15-0be1-4ef0-8d95-bce5f307d3cf None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:10.435778 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36e1a15-0be1-4ef0-8d95-bce5f307d3cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:10.453005 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-d36e1a15-0be1-4ef0-8d95-bce5f307d3cf None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 04:58:10.454542 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-d36e1a15-0be1-4ef0-8d95-bce5f307d3cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 04:58:10.504717 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645d084a-7e35-4a9f-b934-cab7b82557fa None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:10.526879 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 94/177] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 04:58:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 18 04:58:10.528673 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d2d3c8-8f5d-428b-9356-a676f33f3d6f None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:10.529164 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d2d3c8-8f5d-428b-9356-a676f33f3d6f None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:10.529665 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d2d3c8-8f5d-428b-9356-a676f33f3d6f None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:10.531150 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 95/178] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 04:58:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 04:58:10.532056 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d2d3c8-8f5d-428b-9356-a676f33f3d6f None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:10.540477 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd7d2c3-1f20-40a1-803a-8a51a4f40de9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:10.541130 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd7d2c3-1f20-40a1-803a-8a51a4f40de9 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:10.541449 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd7d2c3-1f20-40a1-803a-8a51a4f40de9 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:10.541731 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd7d2c3-1f20-40a1-803a-8a51a4f40de9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:10.542791 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd7d2c3-1f20-40a1-803a-8a51a4f40de9 None admin] RBAC: Adding query filter params (role.id=0089a3aad83c4ae08d8e9d864957285b, scope.project.id=6706fdb556af472c8319dd2a13abe441, user.id=21ca63be55244b159bcc68b518233c91) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:58:10.542899 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd7d2c3-1f20-40a1-803a-8a51a4f40de9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:10.544306 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd7d2c3-1f20-40a1-803a-8a51a4f40de9 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:10.550493 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 84/179] 10.222.0.23 () {56 vars in 1374 bytes} [Sat Jun 18 04:58:10 2022] GET /identity/v3/role_assignments?user.id=21ca63be55244b159bcc68b518233c91&scope.project.id=6706fdb556af472c8319dd2a13abe441&role.id=0089a3aad83c4ae08d8e9d864957285b => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:10.551265 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd7d2c3-1f20-40a1-803a-8a51a4f40de9 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:10.648389 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d2d3c8-8f5d-428b-9356-a676f33f3d6f None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:10.657423 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:10.676636 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:10.689836 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc239233-87f3-4a4c-b6d8-24e3a06feb87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:10.691077 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc239233-87f3-4a4c-b6d8-24e3a06feb87 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:10.691572 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc239233-87f3-4a4c-b6d8-24e3a06feb87 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:10.692080 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc239233-87f3-4a4c-b6d8-24e3a06feb87 None admin] PATH_INFO: `/v3/projects/fa4a0c00884f4f97b9e8fc0496982551` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:10.694777 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd7d2c3-1f20-40a1-803a-8a51a4f40de9 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:10.699109 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc239233-87f3-4a4c-b6d8-24e3a06feb87 None admin] RBAC: Authorizing `identity:get_project(project_id=fa4a0c00884f4f97b9e8fc0496982551)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:10.699109 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc239233-87f3-4a4c-b6d8-24e3a06feb87 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:10.699109 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 96/180] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Jun 18 04:58:10 2022] GET /identity/v3/projects/fa4a0c00884f4f97b9e8fc0496982551 => generated 312 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:10.700233 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df90b283-8ecc-4695-b948-7df4f6d56253 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:10.700712 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df90b283-8ecc-4695-b948-7df4f6d56253 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:10.701200 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df90b283-8ecc-4695-b948-7df4f6d56253 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:10.720137 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-df90b283-8ecc-4695-b948-7df4f6d56253 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 04:58:10.721687 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-df90b283-8ecc-4695-b948-7df4f6d56253 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 04:58:10.737960 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6529d6-ddbe-4ab1-9b25-8b443cdb7b21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:10.738825 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6529d6-ddbe-4ab1-9b25-8b443cdb7b21 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:10.739357 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6529d6-ddbe-4ab1-9b25-8b443cdb7b21 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:10.739629 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6529d6-ddbe-4ab1-9b25-8b443cdb7b21 None admin] PATH_INFO: `/v3/projects/6706fdb556af472c8319dd2a13abe441` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:10.741484 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6529d6-ddbe-4ab1-9b25-8b443cdb7b21 None admin] RBAC: Authorizing `identity:get_project(project_id=6706fdb556af472c8319dd2a13abe441)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:10.746436 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6529d6-ddbe-4ab1-9b25-8b443cdb7b21 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:10.746436 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 85/181] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Jun 18 04:58:10 2022] GET /identity/v3/projects/6706fdb556af472c8319dd2a13abe441 => generated 298 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:10.746658 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6529d6-ddbe-4ab1-9b25-8b443cdb7b21 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:10.860348 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 97/182] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 04:58:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 18 04:58:10.861709 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7bc8c7-ac36-46da-ba69-17ec076ae33e None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:10.862081 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7bc8c7-ac36-46da-ba69-17ec076ae33e None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:10.862357 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7bc8c7-ac36-46da-ba69-17ec076ae33e None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:10.863642 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 98/183] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 04:58:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 04:58:10.864582 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7bc8c7-ac36-46da-ba69-17ec076ae33e None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:10.887286 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6529d6-ddbe-4ab1-9b25-8b443cdb7b21 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:10.921265 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ac9f84-c7fa-48ff-ad1b-ea0422814c51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:10.922013 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ac9f84-c7fa-48ff-ad1b-ea0422814c51 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:10.922299 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ac9f84-c7fa-48ff-ad1b-ea0422814c51 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:10.922573 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ac9f84-c7fa-48ff-ad1b-ea0422814c51 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:10.932696 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ac9f84-c7fa-48ff-ad1b-ea0422814c51 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 18 04:58:10.940270 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application [None req-88ac9f84-c7fa-48ff-ad1b-ea0422814c51 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 18 04:58:10.940270 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:58:10.940270 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:58:10.940270 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:58:10.940270 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:58:10.940270 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:58:10.940270 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:58:10.940270 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:58:10.940270 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:58:10.940270 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:58:10.940270 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:58:10.940270 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:58:10.940270 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 18 04:58:10.940270 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 18 04:58:10.940270 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 18 04:58:10.940270 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 18 04:58:10.941160 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 18 04:58:10.941160 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 18 04:58:10.941160 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 18 04:58:10.941160 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 18 04:58:10.941160 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:58:10.941160 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:58:10.941160 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:58:10.941160 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:58:10.941160 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:58:10.941160 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:58:10.941160 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:58:10.941160 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application with Lock( Jun 18 04:58:10.941160 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:58:10.941160 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:58:10.941160 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:58:10.941160 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:58:10.942270 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:58:10.942270 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:58:10.942270 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:58:10.942270 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:58:10.942270 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 18 04:58:10.942270 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 18 04:58:10.942270 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 18 04:58:10.942270 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 18 04:58:10.942270 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 18 04:58:10.942270 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 18 04:58:10.942270 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 18 04:58:10.942270 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 04:58:10.942270 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 86/184] 10.222.0.23 () {56 vars in 1096 bytes} [Sat Jun 18 04:58:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:58:10.942899 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693ad2db-cb40-4856-bede-02a658c30995 None None] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:10.942970 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693ad2db-cb40-4856-bede-02a658c30995 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:10.943278 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693ad2db-cb40-4856-bede-02a658c30995 None None] PATH_INFO: `/` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:10.945273 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 87/185] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 04:58:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 04:58:10.945840 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693ad2db-cb40-4856-bede-02a658c30995 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:10.961045 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:10.977982 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7bc8c7-ac36-46da-ba69-17ec076ae33e None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:10.980383 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:11.011948 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4306ab25-ac60-4dcf-ad64-11587c92d7f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:11.012598 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4306ab25-ac60-4dcf-ad64-11587c92d7f1 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:11.012879 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4306ab25-ac60-4dcf-ad64-11587c92d7f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:11.013154 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4306ab25-ac60-4dcf-ad64-11587c92d7f1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:11.014018 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4306ab25-ac60-4dcf-ad64-11587c92d7f1 None admin] RBAC: Adding query filter params (role.id=cbf5c4c26898413d93e73b46b41046a3, scope.project.id=fa4a0c00884f4f97b9e8fc0496982551, user.id=21ca63be55244b159bcc68b518233c91) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:58:11.014331 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4306ab25-ac60-4dcf-ad64-11587c92d7f1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:11.015798 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4306ab25-ac60-4dcf-ad64-11587c92d7f1 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:11.022871 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 99/186] 10.222.0.23 () {56 vars in 1374 bytes} [Sat Jun 18 04:58:10 2022] GET /identity/v3/role_assignments?user.id=21ca63be55244b159bcc68b518233c91&scope.project.id=fa4a0c00884f4f97b9e8fc0496982551&role.id=cbf5c4c26898413d93e73b46b41046a3 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:11.024104 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7021af40-97bb-403d-9a70-3f33d0378cee None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:11.024394 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7021af40-97bb-403d-9a70-3f33d0378cee None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:11.024667 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7021af40-97bb-403d-9a70-3f33d0378cee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:11.050755 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-7021af40-97bb-403d-9a70-3f33d0378cee None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 04:58:11.051996 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-7021af40-97bb-403d-9a70-3f33d0378cee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 04:58:11.059867 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693ad2db-cb40-4856-bede-02a658c30995 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:11.099807 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81effd0b-f8c8-49ea-8088-b8f35f331a0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:11.100646 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81effd0b-f8c8-49ea-8088-b8f35f331a0a None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:11.100931 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81effd0b-f8c8-49ea-8088-b8f35f331a0a None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:11.101207 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81effd0b-f8c8-49ea-8088-b8f35f331a0a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:11.102117 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81effd0b-f8c8-49ea-8088-b8f35f331a0a None admin] RBAC: Adding query filter params (role.id=21c65f2c51bd4dd283d13507d2714467, scope.project.id=6706fdb556af472c8319dd2a13abe441, user.id=5d3e83f3ca7347dea1350fa29bac5d9d) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:58:11.102431 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81effd0b-f8c8-49ea-8088-b8f35f331a0a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:11.103901 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81effd0b-f8c8-49ea-8088-b8f35f331a0a None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:11.121431 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 88/187] 10.222.0.23 () {56 vars in 1374 bytes} [Sat Jun 18 04:58:10 2022] GET /identity/v3/role_assignments?user.id=5d3e83f3ca7347dea1350fa29bac5d9d&scope.project.id=6706fdb556af472c8319dd2a13abe441&role.id=21c65f2c51bd4dd283d13507d2714467 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:11.121431 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81effd0b-f8c8-49ea-8088-b8f35f331a0a None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:11.149797 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 100/188] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 04:58:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 18 04:58:11.150912 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7021af40-97bb-403d-9a70-3f33d0378cee None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:11.229216 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:11.265073 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81effd0b-f8c8-49ea-8088-b8f35f331a0a None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:11.302751 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:11.308024 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7021af40-97bb-403d-9a70-3f33d0378cee None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:11.312706 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1bc463-ec65-4bae-8837-a3502f6999db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:11.313551 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1bc463-ec65-4bae-8837-a3502f6999db None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:11.313831 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1bc463-ec65-4bae-8837-a3502f6999db None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:11.314099 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1bc463-ec65-4bae-8837-a3502f6999db None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:11.326206 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1bc463-ec65-4bae-8837-a3502f6999db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 18 04:58:11.334518 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application [None req-6b1bc463-ec65-4bae-8837-a3502f6999db None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 18 04:58:11.334518 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:58:11.334518 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:58:11.334518 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:58:11.334518 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:58:11.334518 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:58:11.334518 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:58:11.334518 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:58:11.334518 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:58:11.334518 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:58:11.334518 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:58:11.334518 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:58:11.334518 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 18 04:58:11.334518 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 18 04:58:11.334518 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 18 04:58:11.334518 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 18 04:58:11.335418 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 18 04:58:11.335418 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 18 04:58:11.335418 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 18 04:58:11.335418 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 18 04:58:11.335418 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:58:11.335418 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:58:11.335418 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:58:11.335418 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:58:11.335418 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:58:11.335418 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:58:11.335418 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:58:11.335418 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application with Lock( Jun 18 04:58:11.335418 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:58:11.335418 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:58:11.335418 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:58:11.335418 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:58:11.336270 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:58:11.336270 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:58:11.336270 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:58:11.336270 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:58:11.336270 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 18 04:58:11.336270 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 18 04:58:11.336270 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 18 04:58:11.336270 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 18 04:58:11.336270 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 18 04:58:11.336270 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 18 04:58:11.336270 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 18 04:58:11.336270 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 04:58:11.336270 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 89/189] 10.222.0.23 () {56 vars in 1114 bytes} [Sat Jun 18 04:58:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:58:11.339893 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5871d2c5-910b-4595-99dc-a84f9b32175a None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:11.339893 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5871d2c5-910b-4595-99dc-a84f9b32175a None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:11.339893 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5871d2c5-910b-4595-99dc-a84f9b32175a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:11.357631 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2ac940-d867-4527-b2b6-2e05b8d6447f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:11.357631 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2ac940-d867-4527-b2b6-2e05b8d6447f None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:11.357631 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2ac940-d867-4527-b2b6-2e05b8d6447f None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:11.357631 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2ac940-d867-4527-b2b6-2e05b8d6447f None admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:11.359033 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2ac940-d867-4527-b2b6-2e05b8d6447f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:58:11.359033 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2ac940-d867-4527-b2b6-2e05b8d6447f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:11.359181 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-5871d2c5-910b-4595-99dc-a84f9b32175a None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 04:58:11.359181 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-5871d2c5-910b-4595-99dc-a84f9b32175a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 04:58:11.359304 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2ac940-d867-4527-b2b6-2e05b8d6447f None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:11.371352 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 101/190] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Jun 18 04:58:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:11.372469 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2ac940-d867-4527-b2b6-2e05b8d6447f None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:11.433582 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 90/191] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 04:58:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 18 04:58:11.434462 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5871d2c5-910b-4595-99dc-a84f9b32175a None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:11.481607 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2ac940-d867-4527-b2b6-2e05b8d6447f None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:11.498494 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:11.516057 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86bd14e-0fb6-40b1-b5bc-96762a4c132a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:11.516847 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86bd14e-0fb6-40b1-b5bc-96762a4c132a None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:11.517133 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86bd14e-0fb6-40b1-b5bc-96762a4c132a None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:11.517432 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86bd14e-0fb6-40b1-b5bc-96762a4c132a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:11.518618 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86bd14e-0fb6-40b1-b5bc-96762a4c132a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:11.520077 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86bd14e-0fb6-40b1-b5bc-96762a4c132a None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:11.526812 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 102/192] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Jun 18 04:58:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:11.526812 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86bd14e-0fb6-40b1-b5bc-96762a4c132a None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:11.533491 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5871d2c5-910b-4595-99dc-a84f9b32175a None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:11.552063 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:11.567235 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a03615c-aad8-4a9d-815e-72dbec31182c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:11.567898 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a03615c-aad8-4a9d-815e-72dbec31182c None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:11.568185 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a03615c-aad8-4a9d-815e-72dbec31182c None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:11.568462 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a03615c-aad8-4a9d-815e-72dbec31182c None admin] PATH_INFO: `/v3/roles` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:11.569960 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a03615c-aad8-4a9d-815e-72dbec31182c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:58:11.570278 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a03615c-aad8-4a9d-815e-72dbec31182c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:11.571669 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a03615c-aad8-4a9d-815e-72dbec31182c None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:11.579733 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 91/193] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Jun 18 04:58:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:11.580514 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a03615c-aad8-4a9d-815e-72dbec31182c None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:11.625544 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86bd14e-0fb6-40b1-b5bc-96762a4c132a None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:11.659384 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357dfb2f-100e-4e4b-a885-9114ceb724ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:11.660248 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357dfb2f-100e-4e4b-a885-9114ceb724ec None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:11.660530 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357dfb2f-100e-4e4b-a885-9114ceb724ec None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:11.660808 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357dfb2f-100e-4e4b-a885-9114ceb724ec None admin] PATH_INFO: `/v3/users/aab53aabef1d4da08e0c843ea04e4197` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:11.674463 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357dfb2f-100e-4e4b-a885-9114ceb724ec None admin] RBAC: Authorizing `identity:get_user(user_id=aab53aabef1d4da08e0c843ea04e4197)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:11.675926 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357dfb2f-100e-4e4b-a885-9114ceb724ec None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:11.677641 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 103/194] 10.222.0.23 () {56 vars in 1177 bytes} [Sat Jun 18 04:58:11 2022] GET /identity/v3/users/aab53aabef1d4da08e0c843ea04e4197 => generated 283 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:11.682599 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357dfb2f-100e-4e4b-a885-9114ceb724ec None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:11.685014 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a03615c-aad8-4a9d-815e-72dbec31182c None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:11.713397 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:11.721081 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645c333b-9405-45f2-aa72-81407bf7f74e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:11.721976 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645c333b-9405-45f2-aa72-81407bf7f74e None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:11.722254 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645c333b-9405-45f2-aa72-81407bf7f74e None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:11.722524 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645c333b-9405-45f2-aa72-81407bf7f74e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:11.733634 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645c333b-9405-45f2-aa72-81407bf7f74e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 18 04:58:11.742483 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application [None req-645c333b-9405-45f2-aa72-81407bf7f74e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 18 04:58:11.742483 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:58:11.742483 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:58:11.742483 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:58:11.742483 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:58:11.742483 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:58:11.742483 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:58:11.742483 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:58:11.742483 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:58:11.742483 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:58:11.742483 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:58:11.742483 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:58:11.742483 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 18 04:58:11.742483 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 18 04:58:11.742483 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 18 04:58:11.742483 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 18 04:58:11.743417 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 18 04:58:11.743417 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 18 04:58:11.743417 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 18 04:58:11.743417 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 18 04:58:11.743417 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:58:11.743417 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:58:11.743417 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:58:11.743417 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:58:11.743417 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:58:11.743417 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:58:11.743417 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:58:11.743417 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application with Lock( Jun 18 04:58:11.743417 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:58:11.743417 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:58:11.743417 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:58:11.743417 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:58:11.744205 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:58:11.744205 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:58:11.744205 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:58:11.744205 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:58:11.744205 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 18 04:58:11.744205 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 18 04:58:11.744205 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 18 04:58:11.744205 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 18 04:58:11.744205 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 18 04:58:11.744205 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 18 04:58:11.744205 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 18 04:58:11.744205 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 04:58:11.744205 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 92/195] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Jun 18 04:58:11 2022] GET /identity/v3/roles/reader => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:58:11.744893 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645c333b-9405-45f2-aa72-81407bf7f74e None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:11.775341 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:11.791533 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357dfb2f-100e-4e4b-a885-9114ceb724ec None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:11.825803 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b6770e-c6c4-4dcf-907f-d9c0e5d871a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:11.826458 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b6770e-c6c4-4dcf-907f-d9c0e5d871a4 None admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:11.826748 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b6770e-c6c4-4dcf-907f-d9c0e5d871a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:11.827019 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b6770e-c6c4-4dcf-907f-d9c0e5d871a4 None admin] PATH_INFO: `/v3/users` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:11.827751 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b6770e-c6c4-4dcf-907f-d9c0e5d871a4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:11.829191 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b6770e-c6c4-4dcf-907f-d9c0e5d871a4 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:11.845733 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645c333b-9405-45f2-aa72-81407bf7f74e None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:11.878909 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321f7b3c-ef3b-4f84-92c4-6cab518636c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:11.879791 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321f7b3c-ef3b-4f84-92c4-6cab518636c3 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:11.880074 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321f7b3c-ef3b-4f84-92c4-6cab518636c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:11.880347 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321f7b3c-ef3b-4f84-92c4-6cab518636c3 None admin] PATH_INFO: `/v3/users/aab53aabef1d4da08e0c843ea04e4197` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:11.882204 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321f7b3c-ef3b-4f84-92c4-6cab518636c3 None admin] RBAC: Authorizing `identity:get_user(user_id=aab53aabef1d4da08e0c843ea04e4197)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:11.883636 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321f7b3c-ef3b-4f84-92c4-6cab518636c3 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:11.885357 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 93/196] 10.222.0.23 () {56 vars in 1177 bytes} [Sat Jun 18 04:58:11 2022] GET /identity/v3/users/aab53aabef1d4da08e0c843ea04e4197 => generated 283 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:11.886140 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321f7b3c-ef3b-4f84-92c4-6cab518636c3 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:11.937448 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 104/197] 10.222.0.23 () {60 vars in 1132 bytes} [Sat Jun 18 04:58:11 2022] POST /identity/v3/users => generated 297 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 04:58:11.938397 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b6770e-c6c4-4dcf-907f-d9c0e5d871a4 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:11.941661 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:11.991660 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321f7b3c-ef3b-4f84-92c4-6cab518636c3 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:12.032102 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3164e161-80e1-4abb-bef5-5bb45b27d10e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:12.032797 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3164e161-80e1-4abb-bef5-5bb45b27d10e None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:12.033082 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3164e161-80e1-4abb-bef5-5bb45b27d10e None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:12.033358 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3164e161-80e1-4abb-bef5-5bb45b27d10e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:12.061435 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b6770e-c6c4-4dcf-907f-d9c0e5d871a4 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:12.100923 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7128dd22-3ce5-4f7e-aa17-ac6af7ac1420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:12.101644 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7128dd22-3ce5-4f7e-aa17-ac6af7ac1420 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:12.101928 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7128dd22-3ce5-4f7e-aa17-ac6af7ac1420 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:12.102207 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7128dd22-3ce5-4f7e-aa17-ac6af7ac1420 None admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:12.102950 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7128dd22-3ce5-4f7e-aa17-ac6af7ac1420 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:58:12.103261 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7128dd22-3ce5-4f7e-aa17-ac6af7ac1420 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:12.104673 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7128dd22-3ce5-4f7e-aa17-ac6af7ac1420 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:12.108994 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3164e161-80e1-4abb-bef5-5bb45b27d10e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:12.110500 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3164e161-80e1-4abb-bef5-5bb45b27d10e None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:12.113311 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 105/198] 10.222.0.23 () {56 vars in 1101 bytes} [Sat Jun 18 04:58:11 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:12.114466 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7128dd22-3ce5-4f7e-aa17-ac6af7ac1420 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:12.121710 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:12.174486 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:12.239930 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 94/199] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Jun 18 04:58:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 18 04:58:12.242658 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3164e161-80e1-4abb-bef5-5bb45b27d10e None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:12.250389 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7128dd22-3ce5-4f7e-aa17-ac6af7ac1420 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:12.292126 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2675c4af-26ae-4dd1-a873-8fc3e33728fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:12.293164 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2675c4af-26ae-4dd1-a873-8fc3e33728fe None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:12.293800 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2675c4af-26ae-4dd1-a873-8fc3e33728fe None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:12.294203 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2675c4af-26ae-4dd1-a873-8fc3e33728fe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:12.379341 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2675c4af-26ae-4dd1-a873-8fc3e33728fe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:12.380712 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3164e161-80e1-4abb-bef5-5bb45b27d10e None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:12.382304 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2675c4af-26ae-4dd1-a873-8fc3e33728fe None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:12.386573 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:12.428426 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2a98be-388d-4626-856d-3a3117a39776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:12.430070 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2a98be-388d-4626-856d-3a3117a39776 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:12.430525 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2a98be-388d-4626-856d-3a3117a39776 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:12.430940 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2a98be-388d-4626-856d-3a3117a39776 None admin] PATH_INFO: `/v3/users/2b6fc67a02584c8cac37b9b3f2205270` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:12.432999 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2a98be-388d-4626-856d-3a3117a39776 None admin] RBAC: Authorizing `identity:get_user(user_id=2b6fc67a02584c8cac37b9b3f2205270)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:12.435029 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2a98be-388d-4626-856d-3a3117a39776 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:12.438415 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 95/200] 10.222.0.23 () {56 vars in 1177 bytes} [Sat Jun 18 04:58:12 2022] GET /identity/v3/users/2b6fc67a02584c8cac37b9b3f2205270 => generated 289 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:12.443315 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2a98be-388d-4626-856d-3a3117a39776 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:12.471933 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:12.520890 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 106/201] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Jun 18 04:58:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 18 04:58:12.522174 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2675c4af-26ae-4dd1-a873-8fc3e33728fe None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:12.552693 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2a98be-388d-4626-856d-3a3117a39776 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:12.600305 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58b38a6-9a38-4db2-9149-d664667be3ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:12.601526 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58b38a6-9a38-4db2-9149-d664667be3ba None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:12.601824 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58b38a6-9a38-4db2-9149-d664667be3ba None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:12.602106 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58b38a6-9a38-4db2-9149-d664667be3ba None admin] PATH_INFO: `/v3/projects/41cb6d0c536f4c4494fdabc03643aef0` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:12.603569 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58b38a6-9a38-4db2-9149-d664667be3ba None admin] RBAC: Authorizing `identity:get_project(project_id=41cb6d0c536f4c4494fdabc03643aef0)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:12.606284 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58b38a6-9a38-4db2-9149-d664667be3ba None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:12.609472 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 96/202] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Jun 18 04:58:12 2022] GET /identity/v3/projects/41cb6d0c536f4c4494fdabc03643aef0 => generated 302 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:12.610411 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58b38a6-9a38-4db2-9149-d664667be3ba None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:12.644310 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2675c4af-26ae-4dd1-a873-8fc3e33728fe None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:12.648398 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:12.683483 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c62b83e-39d5-4481-8405-61ada8034625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:12.686032 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c62b83e-39d5-4481-8405-61ada8034625 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:12.686425 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c62b83e-39d5-4481-8405-61ada8034625 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:12.687691 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c62b83e-39d5-4481-8405-61ada8034625 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:12.735171 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:12.745524 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58b38a6-9a38-4db2-9149-d664667be3ba None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:12.782937 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6736024b-6ffd-4b13-b8cb-5b89f7f8f023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:12.783871 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6736024b-6ffd-4b13-b8cb-5b89f7f8f023 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:12.784319 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6736024b-6ffd-4b13-b8cb-5b89f7f8f023 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:12.784698 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6736024b-6ffd-4b13-b8cb-5b89f7f8f023 None admin] PATH_INFO: `/v3/projects/41cb6d0c536f4c4494fdabc03643aef0` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:12.785163 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c62b83e-39d5-4481-8405-61ada8034625 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:12.786159 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6736024b-6ffd-4b13-b8cb-5b89f7f8f023 None admin] RBAC: Authorizing `identity:get_project(project_id=41cb6d0c536f4c4494fdabc03643aef0)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:12.787068 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c62b83e-39d5-4481-8405-61ada8034625 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:12.787932 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6736024b-6ffd-4b13-b8cb-5b89f7f8f023 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:12.789910 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 97/203] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Jun 18 04:58:12 2022] GET /identity/v3/projects/41cb6d0c536f4c4494fdabc03643aef0 => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:12.794765 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6736024b-6ffd-4b13-b8cb-5b89f7f8f023 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:12.907096 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 107/204] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Jun 18 04:58:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 18 04:58:12.908145 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c62b83e-39d5-4481-8405-61ada8034625 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:12.911621 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:12.913617 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6736024b-6ffd-4b13-b8cb-5b89f7f8f023 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:12.951222 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea28c2d1-ed3d-49d2-9d2a-f8755c0770f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:12.952104 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea28c2d1-ed3d-49d2-9d2a-f8755c0770f4 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:12.953069 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea28c2d1-ed3d-49d2-9d2a-f8755c0770f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:12.953541 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea28c2d1-ed3d-49d2-9d2a-f8755c0770f4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:12.955110 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea28c2d1-ed3d-49d2-9d2a-f8755c0770f4 None admin] RBAC: Adding query filter params (role.id=21c65f2c51bd4dd283d13507d2714467, scope.project.id=41cb6d0c536f4c4494fdabc03643aef0, user.id=aab53aabef1d4da08e0c843ea04e4197) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:58:12.955616 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea28c2d1-ed3d-49d2-9d2a-f8755c0770f4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:12.957344 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea28c2d1-ed3d-49d2-9d2a-f8755c0770f4 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:12.964866 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 98/205] 10.222.0.23 () {56 vars in 1374 bytes} [Sat Jun 18 04:58:12 2022] GET /identity/v3/role_assignments?user.id=aab53aabef1d4da08e0c843ea04e4197&scope.project.id=41cb6d0c536f4c4494fdabc03643aef0&role.id=21c65f2c51bd4dd283d13507d2714467 => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:12.966060 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea28c2d1-ed3d-49d2-9d2a-f8755c0770f4 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:12.995926 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:13.047841 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c62b83e-39d5-4481-8405-61ada8034625 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:13.085284 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea28c2d1-ed3d-49d2-9d2a-f8755c0770f4 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:13.116220 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8bdd98-c13f-4d42-80c5-0b50c3c3c0d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:13.117256 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8bdd98-c13f-4d42-80c5-0b50c3c3c0d4 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:13.117796 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8bdd98-c13f-4d42-80c5-0b50c3c3c0d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:13.118222 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8bdd98-c13f-4d42-80c5-0b50c3c3c0d4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:13.119231 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8bdd98-c13f-4d42-80c5-0b50c3c3c0d4 None admin] RBAC: Adding query filter params (role.id=0089a3aad83c4ae08d8e9d864957285b, scope.project.id=41cb6d0c536f4c4494fdabc03643aef0, user.id=aab53aabef1d4da08e0c843ea04e4197) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:58:13.119733 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8bdd98-c13f-4d42-80c5-0b50c3c3c0d4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:13.121424 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8bdd98-c13f-4d42-80c5-0b50c3c3c0d4 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:13.128883 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 108/206] 10.222.0.23 () {56 vars in 1374 bytes} [Sat Jun 18 04:58:12 2022] GET /identity/v3/role_assignments?user.id=aab53aabef1d4da08e0c843ea04e4197&scope.project.id=41cb6d0c536f4c4494fdabc03643aef0&role.id=0089a3aad83c4ae08d8e9d864957285b => generated 259 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:13.183394 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4a0f4b-f508-4894-955b-6509af8156e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:13.184297 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4a0f4b-f508-4894-955b-6509af8156e6 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:13.184580 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4a0f4b-f508-4894-955b-6509af8156e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:13.184855 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4a0f4b-f508-4894-955b-6509af8156e6 None admin] PATH_INFO: `/v3/projects/6706fdb556af472c8319dd2a13abe441` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:13.190434 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4a0f4b-f508-4894-955b-6509af8156e6 None admin] RBAC: Authorizing `identity:get_project(project_id=6706fdb556af472c8319dd2a13abe441)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:13.191965 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4a0f4b-f508-4894-955b-6509af8156e6 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:13.195527 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 99/207] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Jun 18 04:58:12 2022] GET /identity/v3/projects/6706fdb556af472c8319dd2a13abe441 => generated 298 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:13.201248 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8bdd98-c13f-4d42-80c5-0b50c3c3c0d4 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:13.314117 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:13.358481 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8bdd98-c13f-4d42-80c5-0b50c3c3c0d4 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:13.443036 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2dccd0-0617-4915-a356-c4f1c29cdf6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:13.444871 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2dccd0-0617-4915-a356-c4f1c29cdf6b None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:13.444871 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2dccd0-0617-4915-a356-c4f1c29cdf6b None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:13.445034 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2dccd0-0617-4915-a356-c4f1c29cdf6b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:13.445890 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2dccd0-0617-4915-a356-c4f1c29cdf6b None admin] RBAC: Adding query filter params (role.id=80cac029fc6540658713c822fdd75949, scope.project.id=6706fdb556af472c8319dd2a13abe441, user.id=2b6fc67a02584c8cac37b9b3f2205270) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:58:13.446212 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2dccd0-0617-4915-a356-c4f1c29cdf6b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:13.447849 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2dccd0-0617-4915-a356-c4f1c29cdf6b None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:13.458216 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 109/208] 10.222.0.23 () {56 vars in 1374 bytes} [Sat Jun 18 04:58:13 2022] GET /identity/v3/role_assignments?user.id=2b6fc67a02584c8cac37b9b3f2205270&scope.project.id=6706fdb556af472c8319dd2a13abe441&role.id=80cac029fc6540658713c822fdd75949 => generated 608 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:14.382404 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb279ce5-2add-474e-bfca-7fee57dc0b9f None None] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:14.382931 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb279ce5-2add-474e-bfca-7fee57dc0b9f None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:14.383225 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb279ce5-2add-474e-bfca-7fee57dc0b9f None None] PATH_INFO: `/` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:14.384972 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 100/209] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 04:58:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 04:58:14.392646 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d62166b-95a5-4357-85b5-da77b3ef581f None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:14.393035 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d62166b-95a5-4357-85b5-da77b3ef581f None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:14.393434 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d62166b-95a5-4357-85b5-da77b3ef581f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:14.411284 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-5d62166b-95a5-4357-85b5-da77b3ef581f None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 04:58:14.412699 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-5d62166b-95a5-4357-85b5-da77b3ef581f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 04:58:14.416774 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb74837-5f9c-4a24-ba31-5a466bd68251 None None] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:14.417873 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb74837-5f9c-4a24-ba31-5a466bd68251 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:14.417873 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb74837-5f9c-4a24-ba31-5a466bd68251 None None] PATH_INFO: `/` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:14.418843 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 101/210] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 04:58:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 18 04:58:14.423661 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7756f36-daf4-4a54-8b42-cfac79c9ac8c None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:14.423944 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7756f36-daf4-4a54-8b42-cfac79c9ac8c None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:14.424223 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7756f36-daf4-4a54-8b42-cfac79c9ac8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:14.446235 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-a7756f36-daf4-4a54-8b42-cfac79c9ac8c None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 04:58:14.447562 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-a7756f36-daf4-4a54-8b42-cfac79c9ac8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 04:58:14.486868 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 110/211] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 04:58:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 18 04:58:14.511951 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d62166b-95a5-4357-85b5-da77b3ef581f None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:14.544516 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 102/212] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 04:58:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 18 04:58:14.567727 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7756f36-daf4-4a54-8b42-cfac79c9ac8c None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:14.591703 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:14.625776 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:14.640718 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d62166b-95a5-4357-85b5-da77b3ef581f None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:14.671101 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7756f36-daf4-4a54-8b42-cfac79c9ac8c None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:14.682188 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe54fd71-cb91-4f18-af70-403b25887540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:14.683297 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe54fd71-cb91-4f18-af70-403b25887540 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:14.683842 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe54fd71-cb91-4f18-af70-403b25887540 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:14.684330 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe54fd71-cb91-4f18-af70-403b25887540 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:14.695951 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe54fd71-cb91-4f18-af70-403b25887540 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 18 04:58:14.703911 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-fe54fd71-cb91-4f18-af70-403b25887540 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 18 04:58:14.703911 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:58:14.703911 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:58:14.703911 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:58:14.703911 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:58:14.703911 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:58:14.703911 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:58:14.703911 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:58:14.703911 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:58:14.703911 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:58:14.703911 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:58:14.703911 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:58:14.703911 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 18 04:58:14.703911 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 18 04:58:14.703911 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 18 04:58:14.703911 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 18 04:58:14.704763 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 18 04:58:14.704763 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 18 04:58:14.704763 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 18 04:58:14.704763 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 18 04:58:14.704763 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:58:14.704763 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:58:14.704763 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:58:14.704763 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:58:14.704763 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:58:14.704763 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:58:14.704763 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:58:14.704763 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jun 18 04:58:14.704763 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:58:14.704763 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:58:14.704763 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:58:14.704763 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:58:14.705574 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:58:14.705574 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:58:14.705574 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:58:14.705574 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:58:14.705574 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 18 04:58:14.705574 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 18 04:58:14.705574 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 18 04:58:14.705574 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 18 04:58:14.705574 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 18 04:58:14.705574 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 18 04:58:14.705574 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 18 04:58:14.705574 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 04:58:14.712595 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 111/213] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Jun 18 04:58:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:58:14.713787 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe54fd71-cb91-4f18-af70-403b25887540 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:14.714901 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e554111-2e37-4898-bf72-8cea041a0734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:14.715977 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e554111-2e37-4898-bf72-8cea041a0734 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:14.716453 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e554111-2e37-4898-bf72-8cea041a0734 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:14.717183 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e554111-2e37-4898-bf72-8cea041a0734 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:14.719554 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e554111-2e37-4898-bf72-8cea041a0734 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:14.721221 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e554111-2e37-4898-bf72-8cea041a0734 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:14.728114 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 103/214] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Jun 18 04:58:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 18 04:58:14.728633 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e554111-2e37-4898-bf72-8cea041a0734 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:14.837149 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe54fd71-cb91-4f18-af70-403b25887540 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:14.841905 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:14.853295 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e554111-2e37-4898-bf72-8cea041a0734 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:14.862048 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:14.875068 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259fce42-1adf-434c-bad4-9d061a2624ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:14.876015 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259fce42-1adf-434c-bad4-9d061a2624ac None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:14.876618 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259fce42-1adf-434c-bad4-9d061a2624ac None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:14.877085 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259fce42-1adf-434c-bad4-9d061a2624ac None admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:14.878077 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259fce42-1adf-434c-bad4-9d061a2624ac None admin] RBAC: Adding query filter params (name=member) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:58:14.878639 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259fce42-1adf-434c-bad4-9d061a2624ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:14.880573 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259fce42-1adf-434c-bad4-9d061a2624ac None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:14.891513 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083e0d43-2fad-4db7-ad05-cb8c40395446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:14.891950 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 112/215] 10.222.0.23 () {56 vars in 1101 bytes} [Sat Jun 18 04:58:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:14.896392 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083e0d43-2fad-4db7-ad05-cb8c40395446 None admin] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:14.896392 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083e0d43-2fad-4db7-ad05-cb8c40395446 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:14.896392 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083e0d43-2fad-4db7-ad05-cb8c40395446 None admin] PATH_INFO: `/v3/users` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:14.896392 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083e0d43-2fad-4db7-ad05-cb8c40395446 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:14.896392 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083e0d43-2fad-4db7-ad05-cb8c40395446 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:14.898485 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259fce42-1adf-434c-bad4-9d061a2624ac None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:14.999508 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 104/216] 10.222.0.23 () {60 vars in 1132 bytes} [Sat Jun 18 04:58:14 2022] POST /identity/v3/users => generated 297 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 04:58:15.013637 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259fce42-1adf-434c-bad4-9d061a2624ac None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:15.054144 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489a27a6-d901-4518-92a1-1025e04b997d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:15.055097 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489a27a6-d901-4518-92a1-1025e04b997d None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:15.055389 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489a27a6-d901-4518-92a1-1025e04b997d None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:15.055665 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489a27a6-d901-4518-92a1-1025e04b997d None admin] PATH_INFO: `/v3/users/a616512d0d374729a3eaa676c3469979` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:15.071914 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489a27a6-d901-4518-92a1-1025e04b997d None admin] RBAC: Authorizing `identity:get_user(user_id=a616512d0d374729a3eaa676c3469979)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:15.073425 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489a27a6-d901-4518-92a1-1025e04b997d None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:15.075342 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 113/217] 10.222.0.23 () {56 vars in 1177 bytes} [Sat Jun 18 04:58:14 2022] GET /identity/v3/users/a616512d0d374729a3eaa676c3469979 => generated 297 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:15.081336 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083e0d43-2fad-4db7-ad05-cb8c40395446 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:15.193048 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083e0d43-2fad-4db7-ad05-cb8c40395446 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:15.229290 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855f99f2-39fe-42a8-9f42-d29628b3c401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:15.230035 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855f99f2-39fe-42a8-9f42-d29628b3c401 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:15.230323 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855f99f2-39fe-42a8-9f42-d29628b3c401 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:15.230604 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855f99f2-39fe-42a8-9f42-d29628b3c401 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:15.231421 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe759fe6-c4f7-411f-b556-10939b251176 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:15.231717 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe759fe6-c4f7-411f-b556-10939b251176 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:15.231996 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe759fe6-c4f7-411f-b556-10939b251176 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:15.234194 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 114/218] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 04:58:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 04:58:15.239680 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8379f5df-f3c0-4373-9ed2-f57a3666e298 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:15.239962 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8379f5df-f3c0-4373-9ed2-f57a3666e298 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:15.240235 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8379f5df-f3c0-4373-9ed2-f57a3666e298 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:15.251430 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:15.262819 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-8379f5df-f3c0-4373-9ed2-f57a3666e298 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 04:58:15.264084 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-8379f5df-f3c0-4373-9ed2-f57a3666e298 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 04:58:15.311302 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:15.315477 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855f99f2-39fe-42a8-9f42-d29628b3c401 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:15.316990 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855f99f2-39fe-42a8-9f42-d29628b3c401 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:15.348846 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 115/219] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 04:58:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 18 04:58:15.350511 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc21a9a-0ad6-48b4-95ad-6e1e383c4c12 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:15.350800 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc21a9a-0ad6-48b4-95ad-6e1e383c4c12 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:15.351072 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc21a9a-0ad6-48b4-95ad-6e1e383c4c12 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:15.352545 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 116/220] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 04:58:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 04:58:15.358312 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b266f696-2043-461f-a6a6-96d1a51b5faf None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:15.358601 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b266f696-2043-461f-a6a6-96d1a51b5faf None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:15.358919 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b266f696-2043-461f-a6a6-96d1a51b5faf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:15.381959 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-b266f696-2043-461f-a6a6-96d1a51b5faf None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 04:58:15.384568 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-b266f696-2043-461f-a6a6-96d1a51b5faf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 04:58:15.450152 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 105/221] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Jun 18 04:58:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 18 04:58:15.451634 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855f99f2-39fe-42a8-9f42-d29628b3c401 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:15.465449 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 117/222] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 04:58:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 18 04:58:15.466546 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b266f696-2043-461f-a6a6-96d1a51b5faf None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:15.504210 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:15.562849 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855f99f2-39fe-42a8-9f42-d29628b3c401 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:15.575253 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b266f696-2043-461f-a6a6-96d1a51b5faf None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:15.577494 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:15.602550 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab11038e-59c8-429e-b5f0-83e502918422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:15.603347 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab11038e-59c8-429e-b5f0-83e502918422 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:15.603636 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab11038e-59c8-429e-b5f0-83e502918422 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:15.607026 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab11038e-59c8-429e-b5f0-83e502918422 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:15.616137 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab11038e-59c8-429e-b5f0-83e502918422 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 18 04:58:15.620720 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f657336-24bf-497b-be77-29036d95bce2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:15.621851 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f657336-24bf-497b-be77-29036d95bce2 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:15.622349 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f657336-24bf-497b-be77-29036d95bce2 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:15.622794 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f657336-24bf-497b-be77-29036d95bce2 None admin] PATH_INFO: `/v3/projects/41cb6d0c536f4c4494fdabc03643aef0` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:15.624154 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f657336-24bf-497b-be77-29036d95bce2 None admin] RBAC: Authorizing `identity:get_project(project_id=41cb6d0c536f4c4494fdabc03643aef0)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:15.624393 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application [None req-ab11038e-59c8-429e-b5f0-83e502918422 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 18 04:58:15.624393 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:58:15.624393 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:58:15.624393 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:58:15.624393 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:58:15.624393 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:58:15.624393 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:58:15.624393 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:58:15.624393 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:58:15.624393 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:58:15.624393 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:58:15.624393 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:58:15.624393 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 18 04:58:15.624393 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 18 04:58:15.624393 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 18 04:58:15.624393 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 18 04:58:15.625226 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 18 04:58:15.625226 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 18 04:58:15.625226 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 18 04:58:15.625226 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 18 04:58:15.625226 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:58:15.625226 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:58:15.625226 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:58:15.625226 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:58:15.625226 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:58:15.625226 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:58:15.625226 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:58:15.625226 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application with Lock( Jun 18 04:58:15.625226 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:58:15.625226 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:58:15.625226 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:58:15.625226 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:58:15.626036 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:58:15.626036 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:58:15.626036 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:58:15.626036 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:58:15.626036 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 18 04:58:15.626036 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 18 04:58:15.626036 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 18 04:58:15.626036 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 18 04:58:15.626036 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 18 04:58:15.626036 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 18 04:58:15.626036 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 18 04:58:15.626036 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 04:58:15.626036 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 106/223] 10.222.0.23 () {56 vars in 1096 bytes} [Sat Jun 18 04:58:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:58:15.626680 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab11038e-59c8-429e-b5f0-83e502918422 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:15.628710 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f657336-24bf-497b-be77-29036d95bce2 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:15.630847 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 118/224] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Jun 18 04:58:15 2022] GET /identity/v3/projects/41cb6d0c536f4c4494fdabc03643aef0 => generated 302 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:15.633064 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f657336-24bf-497b-be77-29036d95bce2 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:15.747911 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab11038e-59c8-429e-b5f0-83e502918422 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:15.758068 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:15.781408 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f657336-24bf-497b-be77-29036d95bce2 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:15.792554 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8b67d3-14aa-4948-92c1-a7082a0e9189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:15.793584 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8b67d3-14aa-4948-92c1-a7082a0e9189 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:15.793984 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8b67d3-14aa-4948-92c1-a7082a0e9189 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:15.794360 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8b67d3-14aa-4948-92c1-a7082a0e9189 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:15.807465 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8b67d3-14aa-4948-92c1-a7082a0e9189 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 18 04:58:15.815776 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application [None req-af8b67d3-14aa-4948-92c1-a7082a0e9189 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 18 04:58:15.815776 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:58:15.815776 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:58:15.815776 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:58:15.815776 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:58:15.815776 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:58:15.815776 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:58:15.815776 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:58:15.815776 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:58:15.815776 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:58:15.815776 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:58:15.815776 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:58:15.815776 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 18 04:58:15.815776 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 18 04:58:15.815776 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 18 04:58:15.815776 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 18 04:58:15.816692 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 18 04:58:15.816692 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 18 04:58:15.816692 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 18 04:58:15.816692 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 18 04:58:15.816692 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:58:15.816692 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:58:15.816692 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:58:15.816692 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:58:15.816692 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:58:15.816692 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:58:15.816692 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:58:15.816692 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application with Lock( Jun 18 04:58:15.816692 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:58:15.816692 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:58:15.816692 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:58:15.816692 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:58:15.817502 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:58:15.817502 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:58:15.817502 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:58:15.817502 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:58:15.817502 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 18 04:58:15.817502 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 18 04:58:15.817502 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 18 04:58:15.817502 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 18 04:58:15.817502 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 18 04:58:15.817502 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 18 04:58:15.817502 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 18 04:58:15.817502 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 04:58:15.817502 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 107/225] 10.222.0.23 () {56 vars in 1114 bytes} [Sat Jun 18 04:58:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:58:15.818164 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8b67d3-14aa-4948-92c1-a7082a0e9189 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:15.824202 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0d9edd-cd6c-4cd8-9cb3-9dccc9631a59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:15.825098 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0d9edd-cd6c-4cd8-9cb3-9dccc9631a59 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:15.825583 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0d9edd-cd6c-4cd8-9cb3-9dccc9631a59 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:15.826026 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0d9edd-cd6c-4cd8-9cb3-9dccc9631a59 None admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:15.826964 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0d9edd-cd6c-4cd8-9cb3-9dccc9631a59 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:58:15.827442 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0d9edd-cd6c-4cd8-9cb3-9dccc9631a59 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:15.829100 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0d9edd-cd6c-4cd8-9cb3-9dccc9631a59 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:15.839808 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 119/226] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Jun 18 04:58:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:15.840823 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0d9edd-cd6c-4cd8-9cb3-9dccc9631a59 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:15.875129 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:15.929320 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8b67d3-14aa-4948-92c1-a7082a0e9189 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:15.952124 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0d9edd-cd6c-4cd8-9cb3-9dccc9631a59 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:15.965339 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c1ea4d-ad99-4625-9d13-47f49e354ef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:15.966198 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c1ea4d-ad99-4625-9d13-47f49e354ef3 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:15.966667 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c1ea4d-ad99-4625-9d13-47f49e354ef3 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:15.967023 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c1ea4d-ad99-4625-9d13-47f49e354ef3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:15.967973 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c1ea4d-ad99-4625-9d13-47f49e354ef3 None admin] RBAC: Adding query filter params (role.id=cbf5c4c26898413d93e73b46b41046a3, scope.project.id=41cb6d0c536f4c4494fdabc03643aef0, user.id=a616512d0d374729a3eaa676c3469979) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:58:15.968419 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c1ea4d-ad99-4625-9d13-47f49e354ef3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:15.970012 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c1ea4d-ad99-4625-9d13-47f49e354ef3 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:15.975992 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 108/227] 10.222.0.23 () {56 vars in 1374 bytes} [Sat Jun 18 04:58:15 2022] GET /identity/v3/role_assignments?user.id=a616512d0d374729a3eaa676c3469979&scope.project.id=41cb6d0c536f4c4494fdabc03643aef0&role.id=cbf5c4c26898413d93e73b46b41046a3 => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:15.976944 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c1ea4d-ad99-4625-9d13-47f49e354ef3 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:15.992619 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e4f35d-1761-4322-ae2d-c74e2552c2e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:15.993659 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e4f35d-1761-4322-ae2d-c74e2552c2e6 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:15.994116 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e4f35d-1761-4322-ae2d-c74e2552c2e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:15.994588 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e4f35d-1761-4322-ae2d-c74e2552c2e6 None admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:15.995602 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e4f35d-1761-4322-ae2d-c74e2552c2e6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:58:15.996098 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e4f35d-1761-4322-ae2d-c74e2552c2e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:15.997790 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e4f35d-1761-4322-ae2d-c74e2552c2e6 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:16.009588 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 120/228] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Jun 18 04:58:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:16.023313 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e4f35d-1761-4322-ae2d-c74e2552c2e6 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:16.041829 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:16.129085 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:16.137913 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e4f35d-1761-4322-ae2d-c74e2552c2e6 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:16.175091 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ab85b5-f7f8-48f3-8900-5b8df724aaae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:16.176054 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ab85b5-f7f8-48f3-8900-5b8df724aaae None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:16.176400 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ab85b5-f7f8-48f3-8900-5b8df724aaae None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:16.176680 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c1ea4d-ad99-4625-9d13-47f49e354ef3 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:16.176792 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ab85b5-f7f8-48f3-8900-5b8df724aaae None admin] PATH_INFO: `/v3/users/aab53aabef1d4da08e0c843ea04e4197` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:16.178095 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ab85b5-f7f8-48f3-8900-5b8df724aaae None admin] RBAC: Authorizing `identity:get_user(user_id=aab53aabef1d4da08e0c843ea04e4197)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:16.179643 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ab85b5-f7f8-48f3-8900-5b8df724aaae None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:16.181729 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 121/229] 10.222.0.23 () {56 vars in 1177 bytes} [Sat Jun 18 04:58:16 2022] GET /identity/v3/users/aab53aabef1d4da08e0c843ea04e4197 => generated 283 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:16.192229 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ab85b5-f7f8-48f3-8900-5b8df724aaae None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:16.261944 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af53305-2bda-49be-b317-77c102d67aa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:16.262896 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af53305-2bda-49be-b317-77c102d67aa3 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:16.263184 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af53305-2bda-49be-b317-77c102d67aa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:16.263459 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af53305-2bda-49be-b317-77c102d67aa3 None admin] PATH_INFO: `/v3/users/aab53aabef1d4da08e0c843ea04e4197` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:16.264777 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af53305-2bda-49be-b317-77c102d67aa3 None admin] RBAC: Authorizing `identity:get_user(user_id=aab53aabef1d4da08e0c843ea04e4197)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:16.266367 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af53305-2bda-49be-b317-77c102d67aa3 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:16.269985 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 109/230] 10.222.0.23 () {56 vars in 1177 bytes} [Sat Jun 18 04:58:15 2022] GET /identity/v3/users/aab53aabef1d4da08e0c843ea04e4197 => generated 283 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:16.275416 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af53305-2bda-49be-b317-77c102d67aa3 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:16.316388 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ab85b5-f7f8-48f3-8900-5b8df724aaae None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:16.357411 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642a8a98-e3ad-47ff-8d9f-3c84428453f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:16.358283 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642a8a98-e3ad-47ff-8d9f-3c84428453f1 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:16.358563 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642a8a98-e3ad-47ff-8d9f-3c84428453f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:16.358836 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642a8a98-e3ad-47ff-8d9f-3c84428453f1 None admin] PATH_INFO: `/v3/projects/41cb6d0c536f4c4494fdabc03643aef0` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:16.363989 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642a8a98-e3ad-47ff-8d9f-3c84428453f1 None admin] RBAC: Authorizing `identity:get_project(project_id=41cb6d0c536f4c4494fdabc03643aef0)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:16.365942 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642a8a98-e3ad-47ff-8d9f-3c84428453f1 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:16.368173 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 122/231] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Jun 18 04:58:16 2022] GET /identity/v3/projects/41cb6d0c536f4c4494fdabc03643aef0 => generated 302 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 18 04:58:16.378432 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642a8a98-e3ad-47ff-8d9f-3c84428453f1 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:16.424320 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:16.429213 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af53305-2bda-49be-b317-77c102d67aa3 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:16.438910 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:16.504019 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3067332e-c557-494b-a233-2f0e5945cee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:16.504878 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3067332e-c557-494b-a233-2f0e5945cee8 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:16.505157 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3067332e-c557-494b-a233-2f0e5945cee8 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:16.505464 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3067332e-c557-494b-a233-2f0e5945cee8 None admin] PATH_INFO: `/v3/projects/41cb6d0c536f4c4494fdabc03643aef0` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:16.507083 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3067332e-c557-494b-a233-2f0e5945cee8 None admin] RBAC: Authorizing `identity:get_project(project_id=41cb6d0c536f4c4494fdabc03643aef0)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:16.509970 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3067332e-c557-494b-a233-2f0e5945cee8 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:16.511730 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 110/232] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Jun 18 04:58:16 2022] GET /identity/v3/projects/41cb6d0c536f4c4494fdabc03643aef0 => generated 302 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:16.519071 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3067332e-c557-494b-a233-2f0e5945cee8 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:16.556737 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642a8a98-e3ad-47ff-8d9f-3c84428453f1 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:16.600918 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdac29c-d6ce-4da1-b74b-4e94af23446f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:16.602409 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdac29c-d6ce-4da1-b74b-4e94af23446f None admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:16.602748 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdac29c-d6ce-4da1-b74b-4e94af23446f None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:16.603032 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdac29c-d6ce-4da1-b74b-4e94af23446f None admin] PATH_INFO: `/v3/projects/41cb6d0c536f4c4494fdabc03643aef0/users/aab53aabef1d4da08e0c843ea04e4197/roles/0089a3aad83c4ae08d8e9d864957285b` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:16.606500 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdac29c-d6ce-4da1-b74b-4e94af23446f None admin] RBAC: Authorizing `identity:create_grant(project_id=41cb6d0c536f4c4494fdabc03643aef0, user_id=aab53aabef1d4da08e0c843ea04e4197, role_id=0089a3aad83c4ae08d8e9d864957285b)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:16.608442 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdac29c-d6ce-4da1-b74b-4e94af23446f None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:16.658293 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3067332e-c557-494b-a233-2f0e5945cee8 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:16.676796 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 123/233] 10.222.0.23 () {58 vars in 1439 bytes} [Sat Jun 18 04:58:16 2022] PUT /identity/v3/projects/41cb6d0c536f4c4494fdabc03643aef0/users/aab53aabef1d4da08e0c843ea04e4197/roles/0089a3aad83c4ae08d8e9d864957285b => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 04:58:16.703786 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f9f7e7-f2ce-443c-b721-14b406c65062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:16.704684 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f9f7e7-f2ce-443c-b721-14b406c65062 None admin] REQUEST_METHOD: `PUT` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:16.704965 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f9f7e7-f2ce-443c-b721-14b406c65062 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:16.705241 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f9f7e7-f2ce-443c-b721-14b406c65062 None admin] PATH_INFO: `/v3/projects/41cb6d0c536f4c4494fdabc03643aef0/users/aab53aabef1d4da08e0c843ea04e4197/roles/21c65f2c51bd4dd283d13507d2714467` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:16.707364 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f9f7e7-f2ce-443c-b721-14b406c65062 None admin] RBAC: Authorizing `identity:create_grant(project_id=41cb6d0c536f4c4494fdabc03643aef0, user_id=aab53aabef1d4da08e0c843ea04e4197, role_id=21c65f2c51bd4dd283d13507d2714467)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:16.708937 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f9f7e7-f2ce-443c-b721-14b406c65062 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:16.783481 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 111/234] 10.222.0.23 () {58 vars in 1439 bytes} [Sat Jun 18 04:58:16 2022] PUT /identity/v3/projects/41cb6d0c536f4c4494fdabc03643aef0/users/aab53aabef1d4da08e0c843ea04e4197/roles/21c65f2c51bd4dd283d13507d2714467 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 04:58:17.488942 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035eb556-fed0-4362-9a38-916e6ef0fecb None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:17.489532 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035eb556-fed0-4362-9a38-916e6ef0fecb None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:17.489807 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035eb556-fed0-4362-9a38-916e6ef0fecb None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:17.492125 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 124/235] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 04:58:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 04:58:17.497692 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b16918c-618e-44d3-9728-d6f416cf44d2 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:17.497989 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b16918c-618e-44d3-9728-d6f416cf44d2 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:17.498270 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b16918c-618e-44d3-9728-d6f416cf44d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:17.516481 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-9b16918c-618e-44d3-9728-d6f416cf44d2 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 04:58:17.517727 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-9b16918c-618e-44d3-9728-d6f416cf44d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 04:58:17.540099 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:17.541800 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4cdf7e-5110-4804-bd1f-f72aa3ea6b3a None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:17.542086 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4cdf7e-5110-4804-bd1f-f72aa3ea6b3a None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:17.542362 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4cdf7e-5110-4804-bd1f-f72aa3ea6b3a None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:17.543960 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 125/236] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 04:58:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 04:58:17.548936 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0df7860-a981-473d-a95b-19797369ef61 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:17.549223 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0df7860-a981-473d-a95b-19797369ef61 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:17.549571 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0df7860-a981-473d-a95b-19797369ef61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:17.582693 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-e0df7860-a981-473d-a95b-19797369ef61 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 04:58:17.583922 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-e0df7860-a981-473d-a95b-19797369ef61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 04:58:17.626723 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 112/237] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 04:58:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 18 04:58:17.700190 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 126/238] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 04:58:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 18 04:58:17.702128 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b16918c-618e-44d3-9728-d6f416cf44d2 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:17.737550 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0df7860-a981-473d-a95b-19797369ef61 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:17.763394 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:17.867049 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0df7860-a981-473d-a95b-19797369ef61 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:17.873287 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b16918c-618e-44d3-9728-d6f416cf44d2 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:17.933184 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc5ce3c-e509-4bec-a0f8-5a753eaf88ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:17.933407 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809f0867-a702-41c9-b4ad-99ba7538055e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:17.934009 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809f0867-a702-41c9-b4ad-99ba7538055e None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:17.934116 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc5ce3c-e509-4bec-a0f8-5a753eaf88ed None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:17.934289 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc5ce3c-e509-4bec-a0f8-5a753eaf88ed None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:17.934384 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809f0867-a702-41c9-b4ad-99ba7538055e None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:17.934559 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809f0867-a702-41c9-b4ad-99ba7538055e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:17.934663 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc5ce3c-e509-4bec-a0f8-5a753eaf88ed None admin] PATH_INFO: `/v3/domains/default` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:17.938263 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc5ce3c-e509-4bec-a0f8-5a753eaf88ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:17.939735 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc5ce3c-e509-4bec-a0f8-5a753eaf88ed None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:17.941372 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 127/239] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Jun 18 04:58:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 18 04:58:17.947316 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc5ce3c-e509-4bec-a0f8-5a753eaf88ed None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:17.960230 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809f0867-a702-41c9-b4ad-99ba7538055e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 18 04:58:17.967854 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application [None req-809f0867-a702-41c9-b4ad-99ba7538055e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 18 04:58:17.967854 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:58:17.967854 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:58:17.967854 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:58:17.967854 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:58:17.967854 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:58:17.967854 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:58:17.967854 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:58:17.967854 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:58:17.967854 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:58:17.967854 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:58:17.967854 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:58:17.967854 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 18 04:58:17.967854 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 18 04:58:17.967854 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 18 04:58:17.967854 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 18 04:58:17.968727 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 18 04:58:17.968727 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 18 04:58:17.968727 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 18 04:58:17.968727 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 18 04:58:17.968727 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:58:17.968727 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:58:17.968727 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:58:17.968727 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:58:17.968727 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:58:17.968727 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:58:17.968727 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:58:17.968727 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application with Lock( Jun 18 04:58:17.968727 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:58:17.968727 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:58:17.968727 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:58:17.968727 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:58:17.969561 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:58:17.969561 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:58:17.969561 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:58:17.969561 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:58:17.969561 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 18 04:58:17.969561 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 18 04:58:17.969561 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 18 04:58:17.969561 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 18 04:58:17.969561 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 18 04:58:17.969561 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 18 04:58:17.969561 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 18 04:58:17.969561 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 04:58:17.971282 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 113/240] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Jun 18 04:58:17 2022] GET /identity/v3/roles/reader => generated 84 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:58:17.978375 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809f0867-a702-41c9-b4ad-99ba7538055e None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:18.005172 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:18.104899 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809f0867-a702-41c9-b4ad-99ba7538055e None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:18.143750 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:18.149925 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7193dc-61ac-4b69-99a8-5de2b6ab635d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:18.150594 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7193dc-61ac-4b69-99a8-5de2b6ab635d None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:18.150873 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7193dc-61ac-4b69-99a8-5de2b6ab635d None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:18.151146 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7193dc-61ac-4b69-99a8-5de2b6ab635d None admin] PATH_INFO: `/v3/roles` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:18.151963 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7193dc-61ac-4b69-99a8-5de2b6ab635d None admin] RBAC: Adding query filter params (name=reader) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:58:18.152296 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7193dc-61ac-4b69-99a8-5de2b6ab635d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:18.153927 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7193dc-61ac-4b69-99a8-5de2b6ab635d None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:18.171807 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc5ce3c-e509-4bec-a0f8-5a753eaf88ed None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:18.173333 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 114/241] 10.222.0.23 () {56 vars in 1101 bytes} [Sat Jun 18 04:58:17 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:18.180560 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7193dc-61ac-4b69-99a8-5de2b6ab635d None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:18.244709 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9c6e5d-0f7b-4873-bf51-25b762d269c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:18.245775 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9c6e5d-0f7b-4873-bf51-25b762d269c8 None admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:18.246266 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9c6e5d-0f7b-4873-bf51-25b762d269c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:18.246764 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9c6e5d-0f7b-4873-bf51-25b762d269c8 None admin] PATH_INFO: `/v3/users` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:18.247686 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9c6e5d-0f7b-4873-bf51-25b762d269c8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:18.249473 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9c6e5d-0f7b-4873-bf51-25b762d269c8 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:18.299889 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:18.328325 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 128/242] 10.222.0.23 () {60 vars in 1132 bytes} [Sat Jun 18 04:58:17 2022] POST /identity/v3/users => generated 293 bytes in 381 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 04:58:18.338558 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7193dc-61ac-4b69-99a8-5de2b6ab635d None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:18.387166 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a59cf0f-8fa0-4a88-a557-aadd248d35da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:18.388070 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a59cf0f-8fa0-4a88-a557-aadd248d35da None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:18.388351 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a59cf0f-8fa0-4a88-a557-aadd248d35da None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:18.388622 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a59cf0f-8fa0-4a88-a557-aadd248d35da None admin] PATH_INFO: `/v3/users/9d7277d0fe1f4295a3abfbc6356ac7fe` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:18.412148 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a59cf0f-8fa0-4a88-a557-aadd248d35da None admin] RBAC: Authorizing `identity:get_user(user_id=9d7277d0fe1f4295a3abfbc6356ac7fe)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:18.414616 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a59cf0f-8fa0-4a88-a557-aadd248d35da None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:18.416435 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 115/243] 10.222.0.23 () {56 vars in 1177 bytes} [Sat Jun 18 04:58:18 2022] GET /identity/v3/users/9d7277d0fe1f4295a3abfbc6356ac7fe => generated 297 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:18.422352 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9c6e5d-0f7b-4873-bf51-25b762d269c8 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:18.547538 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ee7dbd-9e80-49c4-8ca7-00f430680cee None None] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:18.548043 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ee7dbd-9e80-49c4-8ca7-00f430680cee None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:18.548043 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ee7dbd-9e80-49c4-8ca7-00f430680cee None None] PATH_INFO: `/` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:18.549577 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 116/244] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 04:58:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 18 04:58:18.564293 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96fabfd-e288-46a0-a480-aefa018b4aa6 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:18.564597 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96fabfd-e288-46a0-a480-aefa018b4aa6 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:18.564852 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96fabfd-e288-46a0-a480-aefa018b4aa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:18.592179 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9c6e5d-0f7b-4873-bf51-25b762d269c8 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:18.595722 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-c96fabfd-e288-46a0-a480-aefa018b4aa6 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 04:58:18.596944 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-c96fabfd-e288-46a0-a480-aefa018b4aa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 04:58:18.644202 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c158592-60bb-4935-8407-027b834b215a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:18.645320 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c158592-60bb-4935-8407-027b834b215a None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:18.645840 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c158592-60bb-4935-8407-027b834b215a None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:18.646337 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c158592-60bb-4935-8407-027b834b215a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:18.666693 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:18.701407 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 117/245] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 04:58:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 18 04:58:18.724371 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96fabfd-e288-46a0-a480-aefa018b4aa6 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:18.745355 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c158592-60bb-4935-8407-027b834b215a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:18.748039 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c158592-60bb-4935-8407-027b834b215a None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:18.822149 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:18.875390 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96fabfd-e288-46a0-a480-aefa018b4aa6 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:18.902494 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 129/246] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Jun 18 04:58:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 481 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 18 04:58:18.917367 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c158592-60bb-4935-8407-027b834b215a None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:18.951340 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463f1f44-eefc-468a-bc75-f8ec0389d2ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:18.952297 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463f1f44-eefc-468a-bc75-f8ec0389d2ab None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:18.952530 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463f1f44-eefc-468a-bc75-f8ec0389d2ab None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:18.952811 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463f1f44-eefc-468a-bc75-f8ec0389d2ab None admin] PATH_INFO: `/v3/roles/member` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:18.964589 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463f1f44-eefc-468a-bc75-f8ec0389d2ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 18 04:58:18.970491 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:18.973416 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application [None req-463f1f44-eefc-468a-bc75-f8ec0389d2ab None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 18 04:58:18.973416 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:58:18.973416 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:58:18.973416 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:58:18.973416 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:58:18.973416 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:58:18.973416 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:58:18.973416 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:58:18.973416 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:58:18.973416 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:58:18.973416 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:58:18.973416 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:58:18.973416 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 18 04:58:18.973416 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 18 04:58:18.973416 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 18 04:58:18.973416 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 18 04:58:18.974294 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 18 04:58:18.974294 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 18 04:58:18.974294 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 18 04:58:18.974294 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 18 04:58:18.974294 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:58:18.974294 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:58:18.974294 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:58:18.974294 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:58:18.974294 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:58:18.974294 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:58:18.974294 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:58:18.974294 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application with Lock( Jun 18 04:58:18.974294 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:58:18.974294 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:58:18.974294 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:58:18.974294 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:58:18.975186 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:58:18.975186 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:58:18.975186 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:58:18.975186 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:58:18.975186 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 18 04:58:18.975186 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 18 04:58:18.975186 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 18 04:58:18.975186 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 18 04:58:18.975186 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 18 04:58:18.975186 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 18 04:58:18.975186 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 18 04:58:18.975186 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 04:58:18.975186 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 118/247] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Jun 18 04:58:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:58:18.982818 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463f1f44-eefc-468a-bc75-f8ec0389d2ab None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:19.064628 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c158592-60bb-4935-8407-027b834b215a None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:19.112360 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeebeb3-a35d-4ece-8e57-e3b88563667d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:19.113609 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeebeb3-a35d-4ece-8e57-e3b88563667d None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:19.113899 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeebeb3-a35d-4ece-8e57-e3b88563667d None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:19.116254 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeebeb3-a35d-4ece-8e57-e3b88563667d None admin] PATH_INFO: `/v3/projects/41cb6d0c536f4c4494fdabc03643aef0` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:19.116254 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbeebeb3-a35d-4ece-8e57-e3b88563667d None admin] RBAC: Authorizing `identity:get_project(project_id=41cb6d0c536f4c4494fdabc03643aef0)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:19.117659 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbeebeb3-a35d-4ece-8e57-e3b88563667d None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:19.124829 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 130/248] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Jun 18 04:58:18 2022] GET /identity/v3/projects/41cb6d0c536f4c4494fdabc03643aef0 => generated 302 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:19.124976 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463f1f44-eefc-468a-bc75-f8ec0389d2ab None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:19.126466 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeebeb3-a35d-4ece-8e57-e3b88563667d None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:19.164046 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:19.180458 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab78141-636c-4f27-a3eb-75db26889c17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:19.181490 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab78141-636c-4f27-a3eb-75db26889c17 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:19.182611 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab78141-636c-4f27-a3eb-75db26889c17 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:19.183099 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab78141-636c-4f27-a3eb-75db26889c17 None admin] PATH_INFO: `/v3/roles` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:19.184041 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab78141-636c-4f27-a3eb-75db26889c17 None admin] RBAC: Adding query filter params (name=member) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:58:19.184976 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab78141-636c-4f27-a3eb-75db26889c17 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:19.187096 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab78141-636c-4f27-a3eb-75db26889c17 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:19.200271 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 119/249] 10.222.0.23 () {56 vars in 1101 bytes} [Sat Jun 18 04:58:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 18 04:58:19.206410 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab78141-636c-4f27-a3eb-75db26889c17 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:19.257609 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeebeb3-a35d-4ece-8e57-e3b88563667d None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:19.274138 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:19.298496 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe354796-27d7-4e31-ab6b-e8436b02a6b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:19.299548 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe354796-27d7-4e31-ab6b-e8436b02a6b5 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:19.300008 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe354796-27d7-4e31-ab6b-e8436b02a6b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:19.300488 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe354796-27d7-4e31-ab6b-e8436b02a6b5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:19.301513 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe354796-27d7-4e31-ab6b-e8436b02a6b5 None admin] RBAC: Adding query filter params (role.id=80cac029fc6540658713c822fdd75949, scope.project.id=41cb6d0c536f4c4494fdabc03643aef0, user.id=9d7277d0fe1f4295a3abfbc6356ac7fe) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:58:19.302005 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe354796-27d7-4e31-ab6b-e8436b02a6b5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:19.304174 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe354796-27d7-4e31-ab6b-e8436b02a6b5 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:19.310919 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 131/250] 10.222.0.23 () {56 vars in 1374 bytes} [Sat Jun 18 04:58:19 2022] GET /identity/v3/role_assignments?user.id=9d7277d0fe1f4295a3abfbc6356ac7fe&scope.project.id=41cb6d0c536f4c4494fdabc03643aef0&role.id=80cac029fc6540658713c822fdd75949 => generated 259 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 18 04:58:19.312801 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc34370c-89f5-4a72-bebe-4914cd369ff5 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:19.314431 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc34370c-89f5-4a72-bebe-4914cd369ff5 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:19.315039 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc34370c-89f5-4a72-bebe-4914cd369ff5 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:19.317292 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 132/251] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 04:58:19 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 04:58:19.324811 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb686401-d8b7-44b7-a055-834cf9346a79 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:19.325428 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb686401-d8b7-44b7-a055-834cf9346a79 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:19.325882 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb686401-d8b7-44b7-a055-834cf9346a79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:19.358763 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-bb686401-d8b7-44b7-a055-834cf9346a79 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 04:58:19.361685 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-bb686401-d8b7-44b7-a055-834cf9346a79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 04:58:19.410735 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab78141-636c-4f27-a3eb-75db26889c17 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:19.489428 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 133/252] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 04:58:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 18 04:58:19.491887 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25055cca-55f7-40bb-a46c-a64f47220a27 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:19.492173 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25055cca-55f7-40bb-a46c-a64f47220a27 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:19.492451 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25055cca-55f7-40bb-a46c-a64f47220a27 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:19.494041 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 134/253] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 04:58:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 04:58:19.498196 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fdb493-40e2-4dd4-ad75-6d5326555c09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:19.499101 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fdb493-40e2-4dd4-ad75-6d5326555c09 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:19.499384 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fdb493-40e2-4dd4-ad75-6d5326555c09 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:19.499658 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fdb493-40e2-4dd4-ad75-6d5326555c09 None admin] PATH_INFO: `/v3/users/a616512d0d374729a3eaa676c3469979` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:19.501017 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fdb493-40e2-4dd4-ad75-6d5326555c09 None admin] RBAC: Authorizing `identity:get_user(user_id=a616512d0d374729a3eaa676c3469979)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:19.502527 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fdb493-40e2-4dd4-ad75-6d5326555c09 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:19.504232 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 120/254] 10.222.0.23 () {56 vars in 1177 bytes} [Sat Jun 18 04:58:19 2022] GET /identity/v3/users/a616512d0d374729a3eaa676c3469979 => generated 297 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:19.510253 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fdb493-40e2-4dd4-ad75-6d5326555c09 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:19.510414 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f07a63-356c-4884-979c-fffbd3ff633d None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:19.510631 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f07a63-356c-4884-979c-fffbd3ff633d None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:19.510958 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f07a63-356c-4884-979c-fffbd3ff633d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:19.536988 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-90f07a63-356c-4884-979c-fffbd3ff633d None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 04:58:19.538582 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-90f07a63-356c-4884-979c-fffbd3ff633d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 04:58:19.596130 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:19.625539 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 135/255] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 04:58:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 18 04:58:19.626785 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f07a63-356c-4884-979c-fffbd3ff633d None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:19.670264 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:19.680449 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fdb493-40e2-4dd4-ad75-6d5326555c09 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:19.730945 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315a9583-92a9-432c-820d-722a9d0064a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:19.731812 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315a9583-92a9-432c-820d-722a9d0064a0 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:19.732094 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315a9583-92a9-432c-820d-722a9d0064a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:19.732367 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315a9583-92a9-432c-820d-722a9d0064a0 None admin] PATH_INFO: `/v3/projects/41cb6d0c536f4c4494fdabc03643aef0` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:19.734169 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315a9583-92a9-432c-820d-722a9d0064a0 None admin] RBAC: Authorizing `identity:get_project(project_id=41cb6d0c536f4c4494fdabc03643aef0)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:19.735672 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315a9583-92a9-432c-820d-722a9d0064a0 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:19.737586 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 121/256] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Jun 18 04:58:19 2022] GET /identity/v3/projects/41cb6d0c536f4c4494fdabc03643aef0 => generated 302 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 18 04:58:19.742096 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315a9583-92a9-432c-820d-722a9d0064a0 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:19.783892 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f07a63-356c-4884-979c-fffbd3ff633d None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:19.841964 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c39023-f95f-44ed-92f8-cf183d692fd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:19.842764 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c39023-f95f-44ed-92f8-cf183d692fd7 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:19.843045 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c39023-f95f-44ed-92f8-cf183d692fd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:19.843322 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c39023-f95f-44ed-92f8-cf183d692fd7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:19.876447 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c39023-f95f-44ed-92f8-cf183d692fd7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 18 04:58:19.884695 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-f8c39023-f95f-44ed-92f8-cf183d692fd7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 18 04:58:19.884695 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:58:19.884695 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:58:19.884695 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:58:19.884695 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:58:19.884695 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:58:19.884695 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:58:19.884695 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:58:19.884695 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:58:19.884695 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:58:19.884695 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:58:19.884695 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:58:19.884695 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 18 04:58:19.884695 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 18 04:58:19.884695 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 18 04:58:19.884695 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 18 04:58:19.885598 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 18 04:58:19.885598 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 18 04:58:19.885598 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 18 04:58:19.885598 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 18 04:58:19.885598 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:58:19.885598 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:58:19.885598 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:58:19.885598 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:58:19.885598 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:58:19.885598 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:58:19.885598 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:58:19.885598 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jun 18 04:58:19.885598 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:58:19.885598 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:58:19.885598 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:58:19.885598 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:58:19.886361 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:58:19.886361 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:58:19.886361 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:58:19.886361 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:58:19.886361 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 18 04:58:19.886361 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 18 04:58:19.886361 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 18 04:58:19.886361 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 18 04:58:19.886361 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 18 04:58:19.886361 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 18 04:58:19.886361 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 18 04:58:19.886361 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 04:58:19.898220 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 136/257] 10.222.0.23 () {56 vars in 1096 bytes} [Sat Jun 18 04:58:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:58:19.898877 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c39023-f95f-44ed-92f8-cf183d692fd7 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:19.899571 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315a9583-92a9-432c-820d-722a9d0064a0 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:19.926273 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:19.939833 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96919532-7c6b-42f2-a358-bc204aa7327d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:19.940658 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96919532-7c6b-42f2-a358-bc204aa7327d None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:19.940946 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96919532-7c6b-42f2-a358-bc204aa7327d None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:19.941231 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96919532-7c6b-42f2-a358-bc204aa7327d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:19.953817 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96919532-7c6b-42f2-a358-bc204aa7327d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 18 04:58:19.963936 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application [None req-96919532-7c6b-42f2-a358-bc204aa7327d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 18 04:58:19.963936 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:58:19.963936 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:58:19.963936 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:58:19.963936 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:58:19.963936 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:58:19.963936 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:58:19.963936 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:58:19.963936 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:58:19.963936 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:58:19.963936 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:58:19.963936 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:58:19.963936 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 18 04:58:19.963936 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 18 04:58:19.963936 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 18 04:58:19.963936 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 18 04:58:19.964862 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 18 04:58:19.964862 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 18 04:58:19.964862 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 18 04:58:19.964862 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 18 04:58:19.964862 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:58:19.964862 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:58:19.964862 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:58:19.964862 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:58:19.964862 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:58:19.964862 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:58:19.964862 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:58:19.964862 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application with Lock( Jun 18 04:58:19.964862 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:58:19.964862 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:58:19.964862 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:58:19.964862 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:58:19.965649 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:58:19.965649 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:58:19.965649 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:58:19.965649 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:58:19.965649 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 18 04:58:19.965649 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 18 04:58:19.965649 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 18 04:58:19.965649 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 18 04:58:19.965649 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 18 04:58:19.965649 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 18 04:58:19.965649 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 18 04:58:19.965649 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 04:58:19.965649 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 122/258] 10.222.0.23 () {56 vars in 1114 bytes} [Sat Jun 18 04:58:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:58:19.966888 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96919532-7c6b-42f2-a358-bc204aa7327d None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:20.082746 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96919532-7c6b-42f2-a358-bc204aa7327d None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:20.132082 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a717fab-1788-4368-9466-930377696b9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:20.132753 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a717fab-1788-4368-9466-930377696b9e None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:20.133040 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a717fab-1788-4368-9466-930377696b9e None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:20.133318 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a717fab-1788-4368-9466-930377696b9e None admin] PATH_INFO: `/v3/roles` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:20.134104 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a717fab-1788-4368-9466-930377696b9e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:58:20.134418 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a717fab-1788-4368-9466-930377696b9e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:20.135810 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a717fab-1788-4368-9466-930377696b9e None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:20.151238 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 123/259] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Jun 18 04:58:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:20.157680 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a717fab-1788-4368-9466-930377696b9e None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:20.161713 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:20.172166 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c39023-f95f-44ed-92f8-cf183d692fd7 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:20.246354 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1382fa-430b-4cb8-80a5-6ac7a3e2d192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:20.247167 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1382fa-430b-4cb8-80a5-6ac7a3e2d192 None admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:20.247456 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1382fa-430b-4cb8-80a5-6ac7a3e2d192 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:20.247734 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1382fa-430b-4cb8-80a5-6ac7a3e2d192 None admin] PATH_INFO: `/v3/projects/41cb6d0c536f4c4494fdabc03643aef0/users/a616512d0d374729a3eaa676c3469979/roles/cbf5c4c26898413d93e73b46b41046a3` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:20.251979 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1382fa-430b-4cb8-80a5-6ac7a3e2d192 None admin] RBAC: Authorizing `identity:create_grant(project_id=41cb6d0c536f4c4494fdabc03643aef0, user_id=a616512d0d374729a3eaa676c3469979, role_id=cbf5c4c26898413d93e73b46b41046a3)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:20.255073 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1382fa-430b-4cb8-80a5-6ac7a3e2d192 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:20.262196 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:20.323016 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a717fab-1788-4368-9466-930377696b9e None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:20.331264 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 137/260] 10.222.0.23 () {58 vars in 1439 bytes} [Sat Jun 18 04:58:19 2022] PUT /identity/v3/projects/41cb6d0c536f4c4494fdabc03643aef0/users/a616512d0d374729a3eaa676c3469979/roles/cbf5c4c26898413d93e73b46b41046a3 => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 04:58:20.332181 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1382fa-430b-4cb8-80a5-6ac7a3e2d192 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:20.417594 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd91fea-8776-4096-b1f7-0a11b5a33764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:20.419324 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd91fea-8776-4096-b1f7-0a11b5a33764 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:20.419929 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd91fea-8776-4096-b1f7-0a11b5a33764 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:20.420397 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd91fea-8776-4096-b1f7-0a11b5a33764 None admin] PATH_INFO: `/v3/roles` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:20.421494 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd91fea-8776-4096-b1f7-0a11b5a33764 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:58:20.422073 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd91fea-8776-4096-b1f7-0a11b5a33764 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:20.423926 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd91fea-8776-4096-b1f7-0a11b5a33764 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:20.467878 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 124/261] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Jun 18 04:58:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:20.473442 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd91fea-8776-4096-b1f7-0a11b5a33764 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:20.576929 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1382fa-430b-4cb8-80a5-6ac7a3e2d192 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:20.622856 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64294c46-c178-49b3-8295-c59179edae0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:20.623751 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64294c46-c178-49b3-8295-c59179edae0e None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:20.624200 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64294c46-c178-49b3-8295-c59179edae0e None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:20.624485 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64294c46-c178-49b3-8295-c59179edae0e None admin] PATH_INFO: `/v3/users/aab53aabef1d4da08e0c843ea04e4197` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:20.625736 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64294c46-c178-49b3-8295-c59179edae0e None admin] RBAC: Authorizing `identity:get_user(user_id=aab53aabef1d4da08e0c843ea04e4197)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:20.627202 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64294c46-c178-49b3-8295-c59179edae0e None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:20.629222 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 138/262] 10.222.0.23 () {56 vars in 1177 bytes} [Sat Jun 18 04:58:20 2022] GET /identity/v3/users/aab53aabef1d4da08e0c843ea04e4197 => generated 283 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:20.634343 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64294c46-c178-49b3-8295-c59179edae0e None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:20.674921 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:20.751580 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd91fea-8776-4096-b1f7-0a11b5a33764 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:20.773778 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:20.828772 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64294c46-c178-49b3-8295-c59179edae0e None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:20.866021 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18821dbe-6897-4926-bbb7-898ac8218071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:20.866854 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18821dbe-6897-4926-bbb7-898ac8218071 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:20.867111 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18821dbe-6897-4926-bbb7-898ac8218071 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:20.867393 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18821dbe-6897-4926-bbb7-898ac8218071 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:20.870664 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dff3f2e-6856-445f-9147-f0e930a41fdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:20.872356 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dff3f2e-6856-445f-9147-f0e930a41fdb None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:20.872639 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dff3f2e-6856-445f-9147-f0e930a41fdb None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:20.872928 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dff3f2e-6856-445f-9147-f0e930a41fdb None admin] PATH_INFO: `/v3/users/aab53aabef1d4da08e0c843ea04e4197` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:20.878701 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dff3f2e-6856-445f-9147-f0e930a41fdb None admin] RBAC: Authorizing `identity:get_user(user_id=aab53aabef1d4da08e0c843ea04e4197)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:20.880166 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dff3f2e-6856-445f-9147-f0e930a41fdb None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:20.882087 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 125/263] 10.222.0.23 () {56 vars in 1177 bytes} [Sat Jun 18 04:58:20 2022] GET /identity/v3/users/aab53aabef1d4da08e0c843ea04e4197 => generated 283 bytes in 409 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 18 04:58:20.887338 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dff3f2e-6856-445f-9147-f0e930a41fdb None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:20.999492 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18821dbe-6897-4926-bbb7-898ac8218071 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:21.000986 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18821dbe-6897-4926-bbb7-898ac8218071 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:21.051236 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:21.092245 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dff3f2e-6856-445f-9147-f0e930a41fdb None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:21.161273 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 139/264] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Jun 18 04:58:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 527 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 18 04:58:21.163473 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac62c6c2-fa89-4ccb-91c9-5f40fd561c29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:21.164043 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac62c6c2-fa89-4ccb-91c9-5f40fd561c29 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:21.164332 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac62c6c2-fa89-4ccb-91c9-5f40fd561c29 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:21.164615 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac62c6c2-fa89-4ccb-91c9-5f40fd561c29 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:21.168346 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18821dbe-6897-4926-bbb7-898ac8218071 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:21.218692 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:21.244994 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac62c6c2-fa89-4ccb-91c9-5f40fd561c29 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:21.247137 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac62c6c2-fa89-4ccb-91c9-5f40fd561c29 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:21.361203 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 126/265] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Jun 18 04:58:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 18 04:58:21.373563 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac62c6c2-fa89-4ccb-91c9-5f40fd561c29 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:21.447549 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18821dbe-6897-4926-bbb7-898ac8218071 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:21.458905 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:21.484274 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac62c6c2-fa89-4ccb-91c9-5f40fd561c29 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:21.510459 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:21.526564 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2b9580-8bf1-448a-8351-762618537f89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:21.527423 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2b9580-8bf1-448a-8351-762618537f89 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:21.527702 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2b9580-8bf1-448a-8351-762618537f89 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:21.527977 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2b9580-8bf1-448a-8351-762618537f89 None admin] PATH_INFO: `/v3/projects/41cb6d0c536f4c4494fdabc03643aef0` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:21.529741 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2b9580-8bf1-448a-8351-762618537f89 None admin] RBAC: Authorizing `identity:get_project(project_id=41cb6d0c536f4c4494fdabc03643aef0)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:21.531204 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2b9580-8bf1-448a-8351-762618537f89 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:21.535013 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 127/266] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Jun 18 04:58:21 2022] GET /identity/v3/projects/41cb6d0c536f4c4494fdabc03643aef0 => generated 302 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 18 04:58:21.542707 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f36ef5-6ccc-4b04-b4d1-14bded23fb0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:21.543582 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f36ef5-6ccc-4b04-b4d1-14bded23fb0a None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:21.543861 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f36ef5-6ccc-4b04-b4d1-14bded23fb0a None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:21.544135 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f36ef5-6ccc-4b04-b4d1-14bded23fb0a None admin] PATH_INFO: `/v3/projects/41cb6d0c536f4c4494fdabc03643aef0` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:21.545636 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2b9580-8bf1-448a-8351-762618537f89 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:21.546826 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f36ef5-6ccc-4b04-b4d1-14bded23fb0a None admin] RBAC: Authorizing `identity:get_project(project_id=41cb6d0c536f4c4494fdabc03643aef0)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:21.548273 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f36ef5-6ccc-4b04-b4d1-14bded23fb0a None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:21.550040 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 140/267] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Jun 18 04:58:21 2022] GET /identity/v3/projects/41cb6d0c536f4c4494fdabc03643aef0 => generated 302 bytes in 382 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:21.557661 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f36ef5-6ccc-4b04-b4d1-14bded23fb0a None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:21.745840 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2b9580-8bf1-448a-8351-762618537f89 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:21.763880 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f36ef5-6ccc-4b04-b4d1-14bded23fb0a None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:21.809859 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ae7e55-3616-4d59-97a9-104ced7bc86e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:21.810517 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ae7e55-3616-4d59-97a9-104ced7bc86e None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:21.810790 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ae7e55-3616-4d59-97a9-104ced7bc86e None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:21.811063 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ae7e55-3616-4d59-97a9-104ced7bc86e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:21.811900 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ae7e55-3616-4d59-97a9-104ced7bc86e None admin] RBAC: Adding query filter params (role.id=0089a3aad83c4ae08d8e9d864957285b, scope.project.id=41cb6d0c536f4c4494fdabc03643aef0, user.id=aab53aabef1d4da08e0c843ea04e4197) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:58:21.812216 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ae7e55-3616-4d59-97a9-104ced7bc86e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:21.813990 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ae7e55-3616-4d59-97a9-104ced7bc86e None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:21.822822 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 128/268] 10.222.0.23 () {56 vars in 1374 bytes} [Sat Jun 18 04:58:21 2022] GET /identity/v3/role_assignments?user.id=aab53aabef1d4da08e0c843ea04e4197&scope.project.id=41cb6d0c536f4c4494fdabc03643aef0&role.id=0089a3aad83c4ae08d8e9d864957285b => generated 608 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:21.824636 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12505d89-79a1-45a9-b97d-eac23d06328d None None] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:21.825130 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12505d89-79a1-45a9-b97d-eac23d06328d None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:21.825672 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12505d89-79a1-45a9-b97d-eac23d06328d None None] PATH_INFO: `/` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:21.827491 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 129/269] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 04:58:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 04:58:21.828904 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece7b429-316a-422d-96d1-fd497a5cb6ee None None] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:21.829465 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece7b429-316a-422d-96d1-fd497a5cb6ee None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:21.829945 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece7b429-316a-422d-96d1-fd497a5cb6ee None None] PATH_INFO: `/` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:21.832308 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 130/270] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 04:58:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 04:58:21.838949 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96aa0976-063c-49b0-902a-fe47a392209f None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:21.839529 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96aa0976-063c-49b0-902a-fe47a392209f None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:21.840026 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96aa0976-063c-49b0-902a-fe47a392209f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:21.853131 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c064ee-97dd-4eec-888d-095f5646a36b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:21.853881 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c064ee-97dd-4eec-888d-095f5646a36b None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:21.854166 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c064ee-97dd-4eec-888d-095f5646a36b None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:21.854443 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c064ee-97dd-4eec-888d-095f5646a36b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:21.855283 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c064ee-97dd-4eec-888d-095f5646a36b None admin] RBAC: Adding query filter params (role.id=21c65f2c51bd4dd283d13507d2714467, scope.project.id=41cb6d0c536f4c4494fdabc03643aef0, user.id=aab53aabef1d4da08e0c843ea04e4197) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:58:21.855596 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c064ee-97dd-4eec-888d-095f5646a36b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:21.857081 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c064ee-97dd-4eec-888d-095f5646a36b None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:21.865922 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-96aa0976-063c-49b0-902a-fe47a392209f None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 04:58:21.867503 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-96aa0976-063c-49b0-902a-fe47a392209f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 04:58:21.874224 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 141/271] 10.222.0.23 () {56 vars in 1374 bytes} [Sat Jun 18 04:58:21 2022] GET /identity/v3/role_assignments?user.id=aab53aabef1d4da08e0c843ea04e4197&scope.project.id=41cb6d0c536f4c4494fdabc03643aef0&role.id=21c65f2c51bd4dd283d13507d2714467 => generated 608 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:21.875538 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82833a82-d70d-4b68-9178-17341527462c None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:21.875829 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82833a82-d70d-4b68-9178-17341527462c None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:21.876109 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82833a82-d70d-4b68-9178-17341527462c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:21.878426 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:21.904904 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-82833a82-d70d-4b68-9178-17341527462c None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 04:58:21.906372 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-82833a82-d70d-4b68-9178-17341527462c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 04:58:22.025474 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:22.036852 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 131/272] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 04:58:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 18 04:58:22.038459 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d4ec36-4568-4eff-8924-e51653c25617 None None] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:22.038748 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d4ec36-4568-4eff-8924-e51653c25617 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:22.039023 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d4ec36-4568-4eff-8924-e51653c25617 None None] PATH_INFO: `/` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:22.042227 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 132/273] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 04:58:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 04:58:22.047024 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9f94b5-2236-41f4-badb-e6acba78826e None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:22.047303 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9f94b5-2236-41f4-badb-e6acba78826e None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:22.047581 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9f94b5-2236-41f4-badb-e6acba78826e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:22.055480 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 142/274] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 04:58:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 18 04:58:22.069307 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82833a82-d70d-4b68-9178-17341527462c None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:22.089685 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-ef9f94b5-2236-41f4-badb-e6acba78826e None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 04:58:22.092538 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-ef9f94b5-2236-41f4-badb-e6acba78826e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 04:58:22.189056 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 133/275] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 04:58:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 18 04:58:22.190253 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9f94b5-2236-41f4-badb-e6acba78826e None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:22.210727 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82833a82-d70d-4b68-9178-17341527462c None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:22.255562 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a5df50-8d08-4dd3-9b9a-84f643a137c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:22.257052 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a5df50-8d08-4dd3-9b9a-84f643a137c7 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:22.257459 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a5df50-8d08-4dd3-9b9a-84f643a137c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:22.257766 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a5df50-8d08-4dd3-9b9a-84f643a137c7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:22.269918 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a5df50-8d08-4dd3-9b9a-84f643a137c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 18 04:58:22.279656 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-d7a5df50-8d08-4dd3-9b9a-84f643a137c7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 18 04:58:22.279656 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:58:22.279656 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:58:22.279656 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:58:22.279656 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:58:22.279656 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:58:22.279656 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:58:22.279656 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:58:22.279656 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:58:22.279656 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:58:22.279656 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:58:22.279656 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:58:22.279656 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 18 04:58:22.279656 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 18 04:58:22.279656 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 18 04:58:22.279656 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 18 04:58:22.280526 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 18 04:58:22.280526 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 18 04:58:22.280526 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 18 04:58:22.280526 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 18 04:58:22.280526 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:58:22.280526 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:58:22.280526 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:58:22.280526 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:58:22.280526 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:58:22.280526 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:58:22.280526 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:58:22.280526 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jun 18 04:58:22.280526 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:58:22.280526 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:58:22.280526 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:58:22.280526 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:58:22.281312 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:58:22.281312 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:58:22.281312 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:58:22.281312 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:58:22.281312 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 18 04:58:22.281312 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 18 04:58:22.281312 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 18 04:58:22.281312 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 18 04:58:22.281312 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 18 04:58:22.281312 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 18 04:58:22.281312 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 18 04:58:22.281312 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 04:58:22.281983 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 143/276] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Jun 18 04:58:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 18 04:58:22.282886 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a5df50-8d08-4dd3-9b9a-84f643a137c7 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:22.294877 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:22.331775 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9f94b5-2236-41f4-badb-e6acba78826e None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:22.332512 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:22.372077 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9673d8-affa-4227-b6cc-67fa3eef1401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:22.373189 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9673d8-affa-4227-b6cc-67fa3eef1401 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:22.373678 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9673d8-affa-4227-b6cc-67fa3eef1401 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:22.374127 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9673d8-affa-4227-b6cc-67fa3eef1401 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:22.375316 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9673d8-affa-4227-b6cc-67fa3eef1401 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:22.376786 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9673d8-affa-4227-b6cc-67fa3eef1401 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:22.378754 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 134/277] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Jun 18 04:58:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:22.379525 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9673d8-affa-4227-b6cc-67fa3eef1401 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:22.400826 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a5df50-8d08-4dd3-9b9a-84f643a137c7 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:22.435974 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b223e283-f835-4b4e-ac58-873a9cb3fe29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:22.437132 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b223e283-f835-4b4e-ac58-873a9cb3fe29 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:22.437814 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b223e283-f835-4b4e-ac58-873a9cb3fe29 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:22.438105 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b223e283-f835-4b4e-ac58-873a9cb3fe29 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:22.452799 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b223e283-f835-4b4e-ac58-873a9cb3fe29 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 18 04:58:22.464029 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-b223e283-f835-4b4e-ac58-873a9cb3fe29 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 18 04:58:22.464029 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:58:22.464029 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:58:22.464029 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:58:22.464029 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:58:22.464029 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:58:22.464029 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:58:22.464029 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:58:22.464029 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:58:22.464029 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:58:22.464029 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:58:22.464029 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:58:22.464029 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 18 04:58:22.464029 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 18 04:58:22.464029 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 18 04:58:22.464029 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 18 04:58:22.464891 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 18 04:58:22.464891 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 18 04:58:22.464891 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 18 04:58:22.464891 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 18 04:58:22.464891 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:58:22.464891 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:58:22.464891 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:58:22.464891 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:58:22.464891 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:58:22.464891 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:58:22.464891 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:58:22.464891 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jun 18 04:58:22.464891 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:58:22.464891 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:58:22.464891 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:58:22.464891 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:58:22.466111 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:58:22.466111 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:58:22.466111 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:58:22.466111 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:58:22.466111 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 18 04:58:22.466111 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 18 04:58:22.466111 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 18 04:58:22.466111 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 18 04:58:22.466111 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 18 04:58:22.466111 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 18 04:58:22.466111 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 18 04:58:22.466111 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 04:58:22.466111 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 144/278] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Jun 18 04:58:22 2022] GET /identity/v3/roles/reader => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:58:22.467047 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b223e283-f835-4b4e-ac58-873a9cb3fe29 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:22.489644 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9673d8-affa-4227-b6cc-67fa3eef1401 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:22.525601 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf24533-30e1-4001-b5d8-ecf716dd78ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:22.526265 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf24533-30e1-4001-b5d8-ecf716dd78ab None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:22.526551 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf24533-30e1-4001-b5d8-ecf716dd78ab None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:22.526834 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf24533-30e1-4001-b5d8-ecf716dd78ab None admin] PATH_INFO: `/v3/roles` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:22.527579 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf24533-30e1-4001-b5d8-ecf716dd78ab None admin] RBAC: Adding query filter params (name=member) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:58:22.527892 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf24533-30e1-4001-b5d8-ecf716dd78ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:22.529314 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf24533-30e1-4001-b5d8-ecf716dd78ab None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:22.531705 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:22.538801 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 135/279] 10.222.0.23 () {56 vars in 1101 bytes} [Sat Jun 18 04:58:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:22.539593 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf24533-30e1-4001-b5d8-ecf716dd78ab None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:22.586479 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b223e283-f835-4b4e-ac58-873a9cb3fe29 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:22.594779 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:22.624651 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bc5e5a-edad-4036-aaf4-0e83f7d727eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:22.625451 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bc5e5a-edad-4036-aaf4-0e83f7d727eb None admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:22.625734 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bc5e5a-edad-4036-aaf4-0e83f7d727eb None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:22.626007 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bc5e5a-edad-4036-aaf4-0e83f7d727eb None admin] PATH_INFO: `/v3/users` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:22.626711 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bc5e5a-edad-4036-aaf4-0e83f7d727eb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:22.628489 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bc5e5a-edad-4036-aaf4-0e83f7d727eb None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:22.645059 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf24533-30e1-4001-b5d8-ecf716dd78ab None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:22.678876 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d26d5e2-578a-4ff3-8484-d1f48c8cc26a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:22.679633 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d26d5e2-578a-4ff3-8484-d1f48c8cc26a None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:22.679921 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d26d5e2-578a-4ff3-8484-d1f48c8cc26a None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:22.680198 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d26d5e2-578a-4ff3-8484-d1f48c8cc26a None admin] PATH_INFO: `/v3/roles` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:22.680941 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d26d5e2-578a-4ff3-8484-d1f48c8cc26a None admin] RBAC: Adding query filter params (name=reader) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:58:22.681253 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d26d5e2-578a-4ff3-8484-d1f48c8cc26a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:22.682678 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d26d5e2-578a-4ff3-8484-d1f48c8cc26a None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:22.690859 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 136/280] 10.222.0.23 () {56 vars in 1101 bytes} [Sat Jun 18 04:58:22 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:22.690859 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d26d5e2-578a-4ff3-8484-d1f48c8cc26a None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:22.732244 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 145/281] 10.222.0.23 () {60 vars in 1132 bytes} [Sat Jun 18 04:58:22 2022] POST /identity/v3/users => generated 293 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 04:58:22.733743 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bc5e5a-edad-4036-aaf4-0e83f7d727eb None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:22.764550 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:22.822513 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d26d5e2-578a-4ff3-8484-d1f48c8cc26a None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:22.866792 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2e78c6-2966-4adb-b780-d93f097d241b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:22.868209 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2e78c6-2966-4adb-b780-d93f097d241b None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:22.868638 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2e78c6-2966-4adb-b780-d93f097d241b None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:22.869151 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2e78c6-2966-4adb-b780-d93f097d241b None admin] PATH_INFO: `/v3/users/11fd155f3b5b4efd9abeb3d4952fe438` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:22.879454 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bc5e5a-edad-4036-aaf4-0e83f7d727eb None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:22.886117 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2e78c6-2966-4adb-b780-d93f097d241b None admin] RBAC: Authorizing `identity:get_user(user_id=11fd155f3b5b4efd9abeb3d4952fe438)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:22.887979 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2e78c6-2966-4adb-b780-d93f097d241b None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:22.890469 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 137/282] 10.222.0.23 () {56 vars in 1177 bytes} [Sat Jun 18 04:58:22 2022] GET /identity/v3/users/11fd155f3b5b4efd9abeb3d4952fe438 => generated 293 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:22.896180 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2e78c6-2966-4adb-b780-d93f097d241b None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:22.923828 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baebfa7-835c-4be0-b9cc-30f6fff59612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:22.924813 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baebfa7-835c-4be0-b9cc-30f6fff59612 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:22.925098 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baebfa7-835c-4be0-b9cc-30f6fff59612 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:22.925397 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baebfa7-835c-4be0-b9cc-30f6fff59612 None admin] PATH_INFO: `/v3/users/9d7277d0fe1f4295a3abfbc6356ac7fe` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:22.926661 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5baebfa7-835c-4be0-b9cc-30f6fff59612 None admin] RBAC: Authorizing `identity:get_user(user_id=9d7277d0fe1f4295a3abfbc6356ac7fe)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:22.928949 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5baebfa7-835c-4be0-b9cc-30f6fff59612 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:22.930911 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 146/283] 10.222.0.23 () {56 vars in 1177 bytes} [Sat Jun 18 04:58:22 2022] GET /identity/v3/users/9d7277d0fe1f4295a3abfbc6356ac7fe => generated 297 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:22.936541 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baebfa7-835c-4be0-b9cc-30f6fff59612 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:22.976568 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:23.014854 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2e78c6-2966-4adb-b780-d93f097d241b None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:23.034303 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:23.057768 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876182b5-abd1-40a2-b599-c088808fa124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:23.059077 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876182b5-abd1-40a2-b599-c088808fa124 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:23.059780 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876182b5-abd1-40a2-b599-c088808fa124 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:23.060253 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876182b5-abd1-40a2-b599-c088808fa124 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:23.061811 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876182b5-abd1-40a2-b599-c088808fa124 None admin] RBAC: Adding query filter params (role.id=cbf5c4c26898413d93e73b46b41046a3, user.id=11fd155f3b5b4efd9abeb3d4952fe438, scope.system=all) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:58:23.062354 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876182b5-abd1-40a2-b599-c088808fa124 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:23.064013 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876182b5-abd1-40a2-b599-c088808fa124 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:23.067249 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baebfa7-835c-4be0-b9cc-30f6fff59612 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:23.071231 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 138/284] 10.222.0.23 () {56 vars in 1308 bytes} [Sat Jun 18 04:58:22 2022] GET /identity/v3/role_assignments?user.id=11fd155f3b5b4efd9abeb3d4952fe438&scope.system=all&role.id=cbf5c4c26898413d93e73b46b41046a3 => generated 226 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 18 04:58:23.106946 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31319a2e-5909-4679-8b79-a0873cf608be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:23.107747 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31319a2e-5909-4679-8b79-a0873cf608be None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:23.108028 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31319a2e-5909-4679-8b79-a0873cf608be None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:23.108307 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31319a2e-5909-4679-8b79-a0873cf608be None admin] PATH_INFO: `/v3/projects/41cb6d0c536f4c4494fdabc03643aef0` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:23.109610 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31319a2e-5909-4679-8b79-a0873cf608be None admin] RBAC: Authorizing `identity:get_project(project_id=41cb6d0c536f4c4494fdabc03643aef0)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:23.111155 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31319a2e-5909-4679-8b79-a0873cf608be None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:23.113332 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 147/285] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Jun 18 04:58:22 2022] GET /identity/v3/projects/41cb6d0c536f4c4494fdabc03643aef0 => generated 302 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 18 04:58:23.119649 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876182b5-abd1-40a2-b599-c088808fa124 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:23.161551 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da273bb-7bb3-49d6-82f6-1d49ad5e73ce None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:23.161998 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da273bb-7bb3-49d6-82f6-1d49ad5e73ce None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:23.162913 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da273bb-7bb3-49d6-82f6-1d49ad5e73ce None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:23.166252 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 148/286] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 04:58:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 18 04:58:23.170760 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509b931e-ec93-45f8-a872-b62a649ea3d9 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:23.171190 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509b931e-ec93-45f8-a872-b62a649ea3d9 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:23.171574 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509b931e-ec93-45f8-a872-b62a649ea3d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:23.194087 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-509b931e-ec93-45f8-a872-b62a649ea3d9 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 04:58:23.195527 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-509b931e-ec93-45f8-a872-b62a649ea3d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 04:58:23.259318 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876182b5-abd1-40a2-b599-c088808fa124 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:23.278105 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 149/287] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 04:58:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 18 04:58:23.299397 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0df1d5b-34dc-4621-af88-0cf13718200c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:23.300282 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0df1d5b-34dc-4621-af88-0cf13718200c None admin] REQUEST_METHOD: `PUT` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:23.300559 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0df1d5b-34dc-4621-af88-0cf13718200c None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:23.300827 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0df1d5b-34dc-4621-af88-0cf13718200c None admin] PATH_INFO: `/v3/projects/41cb6d0c536f4c4494fdabc03643aef0/users/9d7277d0fe1f4295a3abfbc6356ac7fe/roles/80cac029fc6540658713c822fdd75949` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:23.303476 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0df1d5b-34dc-4621-af88-0cf13718200c None admin] RBAC: Authorizing `identity:create_grant(project_id=41cb6d0c536f4c4494fdabc03643aef0, user_id=9d7277d0fe1f4295a3abfbc6356ac7fe, role_id=80cac029fc6540658713c822fdd75949)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:23.304991 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0df1d5b-34dc-4621-af88-0cf13718200c None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:23.306382 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509b931e-ec93-45f8-a872-b62a649ea3d9 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:23.334934 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:23.410573 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509b931e-ec93-45f8-a872-b62a649ea3d9 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:23.418129 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 139/288] 10.222.0.23 () {58 vars in 1439 bytes} [Sat Jun 18 04:58:23 2022] PUT /identity/v3/projects/41cb6d0c536f4c4494fdabc03643aef0/users/9d7277d0fe1f4295a3abfbc6356ac7fe/roles/80cac029fc6540658713c822fdd75949 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 04:58:23.449117 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5707a2-b7e1-4d54-b756-11f7a5a8b54f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:23.449998 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5707a2-b7e1-4d54-b756-11f7a5a8b54f None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:23.450282 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5707a2-b7e1-4d54-b756-11f7a5a8b54f None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:23.450556 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5707a2-b7e1-4d54-b756-11f7a5a8b54f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:23.463948 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5707a2-b7e1-4d54-b756-11f7a5a8b54f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 18 04:58:23.472227 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-cc5707a2-b7e1-4d54-b756-11f7a5a8b54f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 18 04:58:23.472227 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:58:23.472227 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:58:23.472227 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:58:23.472227 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:58:23.472227 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:58:23.472227 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:58:23.472227 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:58:23.472227 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:58:23.472227 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:58:23.472227 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:58:23.472227 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:58:23.472227 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 18 04:58:23.472227 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 18 04:58:23.472227 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 18 04:58:23.472227 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 18 04:58:23.473143 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 18 04:58:23.473143 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 18 04:58:23.473143 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 18 04:58:23.473143 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 18 04:58:23.473143 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:58:23.473143 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:58:23.473143 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:58:23.473143 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:58:23.473143 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:58:23.473143 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:58:23.473143 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:58:23.473143 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jun 18 04:58:23.473143 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:58:23.473143 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:58:23.473143 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:58:23.473143 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:58:23.473990 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:58:23.473990 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:58:23.473990 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:58:23.473990 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:58:23.473990 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 18 04:58:23.473990 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 18 04:58:23.473990 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 18 04:58:23.473990 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 18 04:58:23.473990 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 18 04:58:23.473990 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 18 04:58:23.473990 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 18 04:58:23.473990 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 04:58:23.473990 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 150/289] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Jun 18 04:58:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 18 04:58:23.480472 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0df1d5b-34dc-4621-af88-0cf13718200c None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:23.531939 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:23.594238 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0df1d5b-34dc-4621-af88-0cf13718200c None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:23.631798 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a2ecbc-f5da-4522-9991-e9b4a68ce616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:23.632481 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a2ecbc-f5da-4522-9991-e9b4a68ce616 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:23.632767 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a2ecbc-f5da-4522-9991-e9b4a68ce616 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:23.633046 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a2ecbc-f5da-4522-9991-e9b4a68ce616 None admin] PATH_INFO: `/v3/roles` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:23.633847 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a2ecbc-f5da-4522-9991-e9b4a68ce616 None admin] RBAC: Adding query filter params (name=member) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:58:23.634160 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a2ecbc-f5da-4522-9991-e9b4a68ce616 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:23.635576 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a2ecbc-f5da-4522-9991-e9b4a68ce616 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:23.645148 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 140/290] 10.222.0.23 () {56 vars in 1101 bytes} [Sat Jun 18 04:58:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:23.651227 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5707a2-b7e1-4d54-b756-11f7a5a8b54f None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:23.746577 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:23.759491 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5707a2-b7e1-4d54-b756-11f7a5a8b54f None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:23.795258 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4beff9-17a8-4758-9ed2-340d8b8eb83c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:23.796135 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4beff9-17a8-4758-9ed2-340d8b8eb83c None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:23.796416 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4beff9-17a8-4758-9ed2-340d8b8eb83c None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:23.796687 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4beff9-17a8-4758-9ed2-340d8b8eb83c None admin] PATH_INFO: `/v3/users/a616512d0d374729a3eaa676c3469979` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:23.797942 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4beff9-17a8-4758-9ed2-340d8b8eb83c None admin] RBAC: Authorizing `identity:get_user(user_id=a616512d0d374729a3eaa676c3469979)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:23.799699 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4beff9-17a8-4758-9ed2-340d8b8eb83c None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:23.801533 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 151/291] 10.222.0.23 () {56 vars in 1177 bytes} [Sat Jun 18 04:58:23 2022] GET /identity/v3/users/a616512d0d374729a3eaa676c3469979 => generated 297 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:23.807505 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a2ecbc-f5da-4522-9991-e9b4a68ce616 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:23.914209 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a2ecbc-f5da-4522-9991-e9b4a68ce616 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:23.918692 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:23.949948 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58947c1-2184-444f-8925-af44ffc50f8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:23.950597 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58947c1-2184-444f-8925-af44ffc50f8c None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:23.950882 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58947c1-2184-444f-8925-af44ffc50f8c None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:23.951163 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58947c1-2184-444f-8925-af44ffc50f8c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:24.021459 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58947c1-2184-444f-8925-af44ffc50f8c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:24.023528 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58947c1-2184-444f-8925-af44ffc50f8c None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:24.126534 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 141/292] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Jun 18 04:58:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 18 04:58:24.132229 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4beff9-17a8-4758-9ed2-340d8b8eb83c None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:24.234944 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4beff9-17a8-4758-9ed2-340d8b8eb83c None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:24.270859 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3922c8d-f859-4862-b056-c5a11564b90d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:24.271831 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3922c8d-f859-4862-b056-c5a11564b90d None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:24.272045 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3922c8d-f859-4862-b056-c5a11564b90d None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:24.272323 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3922c8d-f859-4862-b056-c5a11564b90d None admin] PATH_INFO: `/v3/projects/41cb6d0c536f4c4494fdabc03643aef0` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:24.273713 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3922c8d-f859-4862-b056-c5a11564b90d None admin] RBAC: Authorizing `identity:get_project(project_id=41cb6d0c536f4c4494fdabc03643aef0)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:24.275168 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3922c8d-f859-4862-b056-c5a11564b90d None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:24.277040 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 152/293] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Jun 18 04:58:24 2022] GET /identity/v3/projects/41cb6d0c536f4c4494fdabc03643aef0 => generated 302 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:24.294866 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58947c1-2184-444f-8925-af44ffc50f8c None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:24.298097 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:24.419094 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58947c1-2184-444f-8925-af44ffc50f8c None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:24.454915 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c815634-a083-49e2-b83f-594ba5c0fb30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:24.455623 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c815634-a083-49e2-b83f-594ba5c0fb30 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:24.455909 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c815634-a083-49e2-b83f-594ba5c0fb30 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:24.456182 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c815634-a083-49e2-b83f-594ba5c0fb30 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:24.457019 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c815634-a083-49e2-b83f-594ba5c0fb30 None admin] RBAC: Adding query filter params (role.id=cbf5c4c26898413d93e73b46b41046a3, scope.project.id=41cb6d0c536f4c4494fdabc03643aef0, user.id=a616512d0d374729a3eaa676c3469979) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:58:24.457335 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c815634-a083-49e2-b83f-594ba5c0fb30 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:24.458816 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c815634-a083-49e2-b83f-594ba5c0fb30 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:24.466365 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 142/294] 10.222.0.23 () {56 vars in 1374 bytes} [Sat Jun 18 04:58:24 2022] GET /identity/v3/role_assignments?user.id=a616512d0d374729a3eaa676c3469979&scope.project.id=41cb6d0c536f4c4494fdabc03643aef0&role.id=cbf5c4c26898413d93e73b46b41046a3 => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:25.402371 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124e88e9-ea35-4967-9db9-660342634b6e None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:25.402863 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124e88e9-ea35-4967-9db9-660342634b6e None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:25.402938 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124e88e9-ea35-4967-9db9-660342634b6e None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:25.404560 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 153/295] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 04:58:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 04:58:25.409687 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25922403-1f93-4a55-803b-45b54883ff04 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:25.410067 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25922403-1f93-4a55-803b-45b54883ff04 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:25.410309 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25922403-1f93-4a55-803b-45b54883ff04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:25.432227 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-25922403-1f93-4a55-803b-45b54883ff04 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 04:58:25.433514 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-25922403-1f93-4a55-803b-45b54883ff04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 04:58:25.494898 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9f72d6-e082-449e-b456-a1cc1edc3166 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:25.495776 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9f72d6-e082-449e-b456-a1cc1edc3166 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:25.496277 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9f72d6-e082-449e-b456-a1cc1edc3166 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:25.498141 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 154/296] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 04:58:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 04:58:25.501413 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:25.503305 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97373ba9-6778-4c2c-a4f6-cd0066a52877 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:25.503825 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97373ba9-6778-4c2c-a4f6-cd0066a52877 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:25.504301 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97373ba9-6778-4c2c-a4f6-cd0066a52877 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:25.504426 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 143/297] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 04:58:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 18 04:58:25.522744 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-97373ba9-6778-4c2c-a4f6-cd0066a52877 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 04:58:25.524264 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-97373ba9-6778-4c2c-a4f6-cd0066a52877 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 04:58:25.528097 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25922403-1f93-4a55-803b-45b54883ff04 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:25.543499 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:25.610694 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 155/298] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 04:58:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 18 04:58:25.612119 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f2d388-4f7c-4e2f-bae3-9841d64e8675 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:25.612409 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f2d388-4f7c-4e2f-bae3-9841d64e8675 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:25.612686 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f2d388-4f7c-4e2f-bae3-9841d64e8675 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:25.614213 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 156/299] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 04:58:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 04:58:25.626939 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c1e812-22fc-4ab8-ad90-3dfe396d5b4f None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:25.627258 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c1e812-22fc-4ab8-ad90-3dfe396d5b4f None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:25.627500 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c1e812-22fc-4ab8-ad90-3dfe396d5b4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:25.642971 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25922403-1f93-4a55-803b-45b54883ff04 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:25.646267 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-28c1e812-22fc-4ab8-ad90-3dfe396d5b4f None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 04:58:25.648588 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-28c1e812-22fc-4ab8-ad90-3dfe396d5b4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 04:58:25.681181 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ef5dd6-5df2-44e8-99dc-bad3052b2d9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:25.682263 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ef5dd6-5df2-44e8-99dc-bad3052b2d9a None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:25.682547 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ef5dd6-5df2-44e8-99dc-bad3052b2d9a None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:25.682820 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ef5dd6-5df2-44e8-99dc-bad3052b2d9a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:25.694718 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ef5dd6-5df2-44e8-99dc-bad3052b2d9a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 18 04:58:25.702987 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application [None req-b2ef5dd6-5df2-44e8-99dc-bad3052b2d9a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 18 04:58:25.702987 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:58:25.702987 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:58:25.702987 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:58:25.702987 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:58:25.702987 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:58:25.702987 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:58:25.702987 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:58:25.702987 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:58:25.702987 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:58:25.702987 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:58:25.702987 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:58:25.702987 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 18 04:58:25.702987 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 18 04:58:25.702987 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 18 04:58:25.702987 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 18 04:58:25.703859 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 18 04:58:25.703859 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 18 04:58:25.703859 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 18 04:58:25.703859 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 18 04:58:25.703859 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:58:25.703859 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:58:25.703859 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:58:25.703859 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:58:25.703859 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:58:25.703859 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:58:25.703859 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:58:25.703859 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application with Lock( Jun 18 04:58:25.703859 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:58:25.703859 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:58:25.703859 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:58:25.703859 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:58:25.704637 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:58:25.704637 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:58:25.704637 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:58:25.704637 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:58:25.704637 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 18 04:58:25.704637 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 18 04:58:25.704637 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 18 04:58:25.704637 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 18 04:58:25.704637 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 18 04:58:25.704637 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 18 04:58:25.704637 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 18 04:58:25.704637 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 04:58:25.704637 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 144/300] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Jun 18 04:58:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 18 04:58:25.705266 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ef5dd6-5df2-44e8-99dc-bad3052b2d9a None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:25.729563 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 157/301] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 04:58:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 18 04:58:25.729563 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c1e812-22fc-4ab8-ad90-3dfe396d5b4f None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:25.775391 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:25.818255 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ef5dd6-5df2-44e8-99dc-bad3052b2d9a None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:25.834964 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:25.851938 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c1e812-22fc-4ab8-ad90-3dfe396d5b4f None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:25.854311 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19ccf18-8c52-4e9e-8ff3-e80f8daed808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:25.855105 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19ccf18-8c52-4e9e-8ff3-e80f8daed808 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:25.855391 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19ccf18-8c52-4e9e-8ff3-e80f8daed808 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:25.855669 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19ccf18-8c52-4e9e-8ff3-e80f8daed808 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:25.856855 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19ccf18-8c52-4e9e-8ff3-e80f8daed808 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:25.858328 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19ccf18-8c52-4e9e-8ff3-e80f8daed808 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:25.864742 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 145/302] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Jun 18 04:58:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:25.864742 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19ccf18-8c52-4e9e-8ff3-e80f8daed808 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:25.891434 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6de888a-49ff-4b53-8a82-1374c6af21d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:25.892098 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6de888a-49ff-4b53-8a82-1374c6af21d2 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:25.892375 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6de888a-49ff-4b53-8a82-1374c6af21d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:25.892648 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6de888a-49ff-4b53-8a82-1374c6af21d2 None admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:25.893454 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6de888a-49ff-4b53-8a82-1374c6af21d2 None admin] RBAC: Adding query filter params (name=member) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:58:25.893732 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6de888a-49ff-4b53-8a82-1374c6af21d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:25.895091 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6de888a-49ff-4b53-8a82-1374c6af21d2 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:25.903816 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 158/303] 10.222.0.23 () {56 vars in 1101 bytes} [Sat Jun 18 04:58:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:25.906189 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6de888a-49ff-4b53-8a82-1374c6af21d2 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:25.965907 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19ccf18-8c52-4e9e-8ff3-e80f8daed808 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:25.991598 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:26.000300 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9044ff-1ca3-4ea4-ab75-c03d42c1dac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:26.001096 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9044ff-1ca3-4ea4-ab75-c03d42c1dac5 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:26.001564 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9044ff-1ca3-4ea4-ab75-c03d42c1dac5 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:26.001850 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9044ff-1ca3-4ea4-ab75-c03d42c1dac5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:26.005588 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6de888a-49ff-4b53-8a82-1374c6af21d2 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:26.012711 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9044ff-1ca3-4ea4-ab75-c03d42c1dac5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 18 04:58:26.020811 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application [None req-2a9044ff-1ca3-4ea4-ab75-c03d42c1dac5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 18 04:58:26.020811 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:58:26.020811 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:58:26.020811 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:58:26.020811 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:58:26.020811 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:58:26.020811 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:58:26.020811 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:58:26.020811 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:58:26.020811 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:58:26.020811 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:58:26.020811 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:58:26.020811 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 18 04:58:26.020811 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 18 04:58:26.020811 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 18 04:58:26.020811 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 18 04:58:26.021775 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 18 04:58:26.021775 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 18 04:58:26.021775 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 18 04:58:26.021775 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 18 04:58:26.021775 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:58:26.021775 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:58:26.021775 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:58:26.021775 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:58:26.021775 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:58:26.021775 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:58:26.021775 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:58:26.021775 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application with Lock( Jun 18 04:58:26.021775 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:58:26.021775 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:58:26.021775 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:58:26.021775 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:58:26.022651 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:58:26.022651 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:58:26.022651 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:58:26.022651 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:58:26.022651 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 18 04:58:26.022651 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 18 04:58:26.022651 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 18 04:58:26.022651 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 18 04:58:26.022651 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 18 04:58:26.022651 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 18 04:58:26.022651 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 18 04:58:26.022651 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 04:58:26.022651 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 146/304] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Jun 18 04:58:25 2022] GET /identity/v3/roles/reader => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:58:26.024017 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc693750-d65f-4ad8-b46b-dff189bee5b6 None None] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:26.024128 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc693750-d65f-4ad8-b46b-dff189bee5b6 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:26.024356 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc693750-d65f-4ad8-b46b-dff189bee5b6 None None] PATH_INFO: `/` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:26.025924 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 147/305] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 04:58:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 04:58:26.029026 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc693750-d65f-4ad8-b46b-dff189bee5b6 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:26.042546 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ef0b66-5622-4276-89a4-bc8d1b9141b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:26.043203 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ef0b66-5622-4276-89a4-bc8d1b9141b7 None admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:26.043493 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ef0b66-5622-4276-89a4-bc8d1b9141b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:26.043766 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ef0b66-5622-4276-89a4-bc8d1b9141b7 None admin] PATH_INFO: `/v3/groups` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:26.044443 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ef0b66-5622-4276-89a4-bc8d1b9141b7 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:26.045917 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ef0b66-5622-4276-89a4-bc8d1b9141b7 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:26.112973 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 159/306] 10.222.0.23 () {60 vars in 1134 bytes} [Sat Jun 18 04:58:25 2022] POST /identity/v3/groups => generated 229 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 04:58:26.113835 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ef0b66-5622-4276-89a4-bc8d1b9141b7 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:26.128854 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:26.135916 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc693750-d65f-4ad8-b46b-dff189bee5b6 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:26.176493 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642f9d0b-6d0b-4ad5-aae0-678c62ee966b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:26.177498 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642f9d0b-6d0b-4ad5-aae0-678c62ee966b None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:26.177799 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642f9d0b-6d0b-4ad5-aae0-678c62ee966b None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:26.178749 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642f9d0b-6d0b-4ad5-aae0-678c62ee966b None admin] PATH_INFO: `/v3/users/11fd155f3b5b4efd9abeb3d4952fe438` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:26.180677 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642f9d0b-6d0b-4ad5-aae0-678c62ee966b None admin] RBAC: Authorizing `identity:get_user(user_id=11fd155f3b5b4efd9abeb3d4952fe438)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:26.182188 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642f9d0b-6d0b-4ad5-aae0-678c62ee966b None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:26.184050 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 148/307] 10.222.0.23 () {56 vars in 1177 bytes} [Sat Jun 18 04:58:26 2022] GET /identity/v3/users/11fd155f3b5b4efd9abeb3d4952fe438 => generated 293 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:26.185362 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937d2a3f-1281-46f2-aede-3152f950d15c None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:26.185670 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937d2a3f-1281-46f2-aede-3152f950d15c None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:26.185944 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937d2a3f-1281-46f2-aede-3152f950d15c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:26.204190 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-937d2a3f-1281-46f2-aede-3152f950d15c None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 04:58:26.205431 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-937d2a3f-1281-46f2-aede-3152f950d15c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 04:58:26.241478 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ef0b66-5622-4276-89a4-bc8d1b9141b7 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:26.263549 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:26.280795 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c27450-4850-4c4b-b13f-e4e3fe51c213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:26.281805 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c27450-4850-4c4b-b13f-e4e3fe51c213 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:26.282090 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c27450-4850-4c4b-b13f-e4e3fe51c213 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:26.282369 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c27450-4850-4c4b-b13f-e4e3fe51c213 None admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:26.283400 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c27450-4850-4c4b-b13f-e4e3fe51c213 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:58:26.283400 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c27450-4850-4c4b-b13f-e4e3fe51c213 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:26.284894 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 149/308] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 04:58:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 18 04:58:26.285004 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c27450-4850-4c4b-b13f-e4e3fe51c213 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:26.285238 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937d2a3f-1281-46f2-aede-3152f950d15c None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:26.294667 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 160/309] 10.222.0.23 () {56 vars in 1101 bytes} [Sat Jun 18 04:58:26 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:26.299654 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c27450-4850-4c4b-b13f-e4e3fe51c213 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:26.378420 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:26.403958 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937d2a3f-1281-46f2-aede-3152f950d15c None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:26.416502 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c27450-4850-4c4b-b13f-e4e3fe51c213 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:26.445815 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd409f02-e0bf-46d3-a084-cf4a2c2adfdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:26.446642 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd409f02-e0bf-46d3-a084-cf4a2c2adfdf None admin] REQUEST_METHOD: `PUT` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:26.446885 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd409f02-e0bf-46d3-a084-cf4a2c2adfdf None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:26.447160 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd409f02-e0bf-46d3-a084-cf4a2c2adfdf None admin] PATH_INFO: `/v3/system/users/11fd155f3b5b4efd9abeb3d4952fe438/roles/cbf5c4c26898413d93e73b46b41046a3` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:26.448641 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd409f02-e0bf-46d3-a084-cf4a2c2adfdf None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=11fd155f3b5b4efd9abeb3d4952fe438, role_id=cbf5c4c26898413d93e73b46b41046a3)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:26.450042 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd409f02-e0bf-46d3-a084-cf4a2c2adfdf None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:26.451340 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32d969d-965e-46f2-86a0-e8ac4d3fdc67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:26.452289 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32d969d-965e-46f2-86a0-e8ac4d3fdc67 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:26.452630 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32d969d-965e-46f2-86a0-e8ac4d3fdc67 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:26.452974 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32d969d-965e-46f2-86a0-e8ac4d3fdc67 None admin] PATH_INFO: `/v3/users/62d3bf53205f44f781ca83e32e58f523` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:26.468100 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32d969d-965e-46f2-86a0-e8ac4d3fdc67 None admin] RBAC: Authorizing `identity:get_user(user_id=62d3bf53205f44f781ca83e32e58f523)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:26.469683 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32d969d-965e-46f2-86a0-e8ac4d3fdc67 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:26.471444 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 161/310] 10.222.0.23 () {56 vars in 1177 bytes} [Sat Jun 18 04:58:26 2022] GET /identity/v3/users/62d3bf53205f44f781ca83e32e58f523 => generated 293 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 18 04:58:26.476231 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32d969d-965e-46f2-86a0-e8ac4d3fdc67 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:26.541478 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 150/311] 10.222.0.23 () {58 vars in 1334 bytes} [Sat Jun 18 04:58:26 2022] PUT /identity/v3/system/users/11fd155f3b5b4efd9abeb3d4952fe438/roles/cbf5c4c26898413d93e73b46b41046a3 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 04:58:26.542397 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd409f02-e0bf-46d3-a084-cf4a2c2adfdf None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:26.603232 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:26.604933 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32d969d-965e-46f2-86a0-e8ac4d3fdc67 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:26.645886 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:26.657918 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bec1ba-0e0c-410d-b96b-09097653a6ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:26.658793 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bec1ba-0e0c-410d-b96b-09097653a6ff None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:26.659081 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bec1ba-0e0c-410d-b96b-09097653a6ff None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:26.659362 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bec1ba-0e0c-410d-b96b-09097653a6ff None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:26.671288 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd409f02-e0bf-46d3-a084-cf4a2c2adfdf None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:26.675600 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bec1ba-0e0c-410d-b96b-09097653a6ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 18 04:58:26.685777 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-88bec1ba-0e0c-410d-b96b-09097653a6ff None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 18 04:58:26.685777 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:58:26.685777 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:58:26.685777 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:58:26.685777 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:58:26.685777 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:58:26.685777 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:58:26.685777 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:58:26.685777 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:58:26.685777 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:58:26.685777 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:58:26.685777 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:58:26.685777 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 18 04:58:26.685777 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 18 04:58:26.685777 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 18 04:58:26.685777 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 18 04:58:26.686733 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 18 04:58:26.686733 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 18 04:58:26.686733 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 18 04:58:26.686733 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 18 04:58:26.686733 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:58:26.686733 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:58:26.686733 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:58:26.686733 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:58:26.686733 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:58:26.686733 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:58:26.686733 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:58:26.686733 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jun 18 04:58:26.686733 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:58:26.686733 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:58:26.686733 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:58:26.686733 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:58:26.687498 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:58:26.687498 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:58:26.687498 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:58:26.687498 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:58:26.687498 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 18 04:58:26.687498 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 18 04:58:26.687498 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 18 04:58:26.687498 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 18 04:58:26.687498 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 18 04:58:26.687498 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 18 04:58:26.687498 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 18 04:58:26.687498 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 04:58:26.687498 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 162/312] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Jun 18 04:58:26 2022] GET /identity/v3/roles/reader => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:58:26.694681 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bec1ba-0e0c-410d-b96b-09097653a6ff None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:26.720669 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b2923c-0b61-486b-83df-e4530a4ea23a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:26.721249 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b2923c-0b61-486b-83df-e4530a4ea23a None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:26.721969 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b2923c-0b61-486b-83df-e4530a4ea23a None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:26.722274 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b2923c-0b61-486b-83df-e4530a4ea23a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:26.723198 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b2923c-0b61-486b-83df-e4530a4ea23a None admin] RBAC: Adding query filter params (role.id=80cac029fc6540658713c822fdd75949, user.id=62d3bf53205f44f781ca83e32e58f523, scope.system=all) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:58:26.723513 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b2923c-0b61-486b-83df-e4530a4ea23a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:26.726281 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b2923c-0b61-486b-83df-e4530a4ea23a None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:26.731598 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 151/313] 10.222.0.23 () {56 vars in 1308 bytes} [Sat Jun 18 04:58:26 2022] GET /identity/v3/role_assignments?user.id=62d3bf53205f44f781ca83e32e58f523&scope.system=all&role.id=80cac029fc6540658713c822fdd75949 => generated 226 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:26.824963 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bec1ba-0e0c-410d-b96b-09097653a6ff None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:26.862035 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30414f05-e396-4d72-8a5e-4fdd0173c6ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:26.862709 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30414f05-e396-4d72-8a5e-4fdd0173c6ee None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:26.862988 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30414f05-e396-4d72-8a5e-4fdd0173c6ee None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:26.863257 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30414f05-e396-4d72-8a5e-4fdd0173c6ee None admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:26.864007 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30414f05-e396-4d72-8a5e-4fdd0173c6ee None admin] RBAC: Adding query filter params (name=reader) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:58:26.864558 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30414f05-e396-4d72-8a5e-4fdd0173c6ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:26.866712 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30414f05-e396-4d72-8a5e-4fdd0173c6ee None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:26.876757 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 163/314] 10.222.0.23 () {56 vars in 1101 bytes} [Sat Jun 18 04:58:26 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 18 04:58:26.882193 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b2923c-0b61-486b-83df-e4530a4ea23a None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:26.997549 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b2923c-0b61-486b-83df-e4530a4ea23a None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:27.017753 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:27.032460 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92d82b1-996f-4ff0-8317-6a102eae1c49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:27.033334 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92d82b1-996f-4ff0-8317-6a102eae1c49 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:27.033643 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92d82b1-996f-4ff0-8317-6a102eae1c49 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:27.033915 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92d82b1-996f-4ff0-8317-6a102eae1c49 None admin] PATH_INFO: `/v3/users/9d7277d0fe1f4295a3abfbc6356ac7fe` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:27.035118 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92d82b1-996f-4ff0-8317-6a102eae1c49 None admin] RBAC: Authorizing `identity:get_user(user_id=9d7277d0fe1f4295a3abfbc6356ac7fe)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:27.036557 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92d82b1-996f-4ff0-8317-6a102eae1c49 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:27.038960 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 152/315] 10.222.0.23 () {56 vars in 1177 bytes} [Sat Jun 18 04:58:26 2022] GET /identity/v3/users/9d7277d0fe1f4295a3abfbc6356ac7fe => generated 297 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:27.043993 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30414f05-e396-4d72-8a5e-4fdd0173c6ee None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:27.104238 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:27.161766 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30414f05-e396-4d72-8a5e-4fdd0173c6ee None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:27.197558 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c6ea04-1343-47f3-96fe-0a3b43400bc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:27.198207 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c6ea04-1343-47f3-96fe-0a3b43400bc5 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:27.198476 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c6ea04-1343-47f3-96fe-0a3b43400bc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:27.198748 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c6ea04-1343-47f3-96fe-0a3b43400bc5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:27.273666 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c6ea04-1343-47f3-96fe-0a3b43400bc5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:27.275614 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c6ea04-1343-47f3-96fe-0a3b43400bc5 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:27.326588 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:27.378944 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 164/316] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Jun 18 04:58:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 18 04:58:27.390766 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92d82b1-996f-4ff0-8317-6a102eae1c49 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:27.500859 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92d82b1-996f-4ff0-8317-6a102eae1c49 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:27.536557 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3800c3e-acfb-4dcd-bed5-46ea6428449a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:27.538054 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3800c3e-acfb-4dcd-bed5-46ea6428449a None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:27.538336 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3800c3e-acfb-4dcd-bed5-46ea6428449a None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:27.538611 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3800c3e-acfb-4dcd-bed5-46ea6428449a None admin] PATH_INFO: `/v3/projects/41cb6d0c536f4c4494fdabc03643aef0` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:27.539805 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3800c3e-acfb-4dcd-bed5-46ea6428449a None admin] RBAC: Authorizing `identity:get_project(project_id=41cb6d0c536f4c4494fdabc03643aef0)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:27.541246 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3800c3e-acfb-4dcd-bed5-46ea6428449a None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:27.543136 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 153/317] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Jun 18 04:58:27 2022] GET /identity/v3/projects/41cb6d0c536f4c4494fdabc03643aef0 => generated 302 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:27.549295 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c6ea04-1343-47f3-96fe-0a3b43400bc5 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:27.657170 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c6ea04-1343-47f3-96fe-0a3b43400bc5 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:27.692474 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb478f78-e867-4009-82f2-2b94124de4df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:27.693132 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb478f78-e867-4009-82f2-2b94124de4df None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:27.693497 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb478f78-e867-4009-82f2-2b94124de4df None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:27.693885 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb478f78-e867-4009-82f2-2b94124de4df None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:27.694723 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb478f78-e867-4009-82f2-2b94124de4df None admin] RBAC: Adding query filter params (role.id=80cac029fc6540658713c822fdd75949, scope.project.id=41cb6d0c536f4c4494fdabc03643aef0, user.id=9d7277d0fe1f4295a3abfbc6356ac7fe) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:58:27.695039 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb478f78-e867-4009-82f2-2b94124de4df None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:27.696483 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb478f78-e867-4009-82f2-2b94124de4df None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:27.702918 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 165/318] 10.222.0.23 () {56 vars in 1374 bytes} [Sat Jun 18 04:58:27 2022] GET /identity/v3/role_assignments?user.id=9d7277d0fe1f4295a3abfbc6356ac7fe&scope.project.id=41cb6d0c536f4c4494fdabc03643aef0&role.id=80cac029fc6540658713c822fdd75949 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:28.211737 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a66566-7f1a-47ca-bf5d-1ff5ca9025e0 None None] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:28.212130 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a66566-7f1a-47ca-bf5d-1ff5ca9025e0 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:28.212304 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a66566-7f1a-47ca-bf5d-1ff5ca9025e0 None None] PATH_INFO: `/` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:28.213742 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 154/319] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 04:58:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 04:58:28.218756 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d787646-7e5b-404e-a5c0-00062d3e3cfe None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:28.219050 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d787646-7e5b-404e-a5c0-00062d3e3cfe None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:28.219325 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d787646-7e5b-404e-a5c0-00062d3e3cfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:28.235797 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-8d787646-7e5b-404e-a5c0-00062d3e3cfe None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 04:58:28.236970 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-8d787646-7e5b-404e-a5c0-00062d3e3cfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 04:58:28.245757 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:28.304945 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 166/320] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 04:58:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 18 04:58:28.327852 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a66566-7f1a-47ca-bf5d-1ff5ca9025e0 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:28.381874 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:28.441248 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a66566-7f1a-47ca-bf5d-1ff5ca9025e0 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:28.477156 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaaa7e0-ea48-4fdf-b8e4-b0edd2fd8150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:28.477896 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaaa7e0-ea48-4fdf-b8e4-b0edd2fd8150 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:28.478182 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaaa7e0-ea48-4fdf-b8e4-b0edd2fd8150 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:28.478453 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaaa7e0-ea48-4fdf-b8e4-b0edd2fd8150 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:28.479639 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeaaa7e0-ea48-4fdf-b8e4-b0edd2fd8150 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:28.481090 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeaaa7e0-ea48-4fdf-b8e4-b0edd2fd8150 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:28.487230 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 155/321] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Jun 18 04:58:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:28.488276 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d787646-7e5b-404e-a5c0-00062d3e3cfe None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:28.590974 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d787646-7e5b-404e-a5c0-00062d3e3cfe None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:28.626887 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73d8be0-9ad7-4728-9b61-7f955a9f5040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:28.627563 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73d8be0-9ad7-4728-9b61-7f955a9f5040 None admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:28.627848 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73d8be0-9ad7-4728-9b61-7f955a9f5040 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:28.628125 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73d8be0-9ad7-4728-9b61-7f955a9f5040 None admin] PATH_INFO: `/v3/groups` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:28.628827 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73d8be0-9ad7-4728-9b61-7f955a9f5040 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:28.630313 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73d8be0-9ad7-4728-9b61-7f955a9f5040 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:28.733503 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 167/322] 10.222.0.23 () {60 vars in 1134 bytes} [Sat Jun 18 04:58:28 2022] POST /identity/v3/groups => generated 226 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 18 04:58:28.747887 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28569619-a00f-425f-b577-5616c0372031 None None] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:28.748402 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28569619-a00f-425f-b577-5616c0372031 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:28.748473 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28569619-a00f-425f-b577-5616c0372031 None None] PATH_INFO: `/` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:28.749883 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 156/323] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 04:58:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 04:58:28.755807 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38a9a9d-06dd-4887-8dea-b43bd31281ed None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:28.756781 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38a9a9d-06dd-4887-8dea-b43bd31281ed None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:28.757203 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38a9a9d-06dd-4887-8dea-b43bd31281ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:28.762800 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:28.776345 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-b38a9a9d-06dd-4887-8dea-b43bd31281ed None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 04:58:28.777599 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-b38a9a9d-06dd-4887-8dea-b43bd31281ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 04:58:28.851248 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 168/324] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 04:58:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 18 04:58:28.875307 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28569619-a00f-425f-b577-5616c0372031 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:28.879429 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cb65ed-f890-41b2-8908-d0f299b6ba95 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:28.879731 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cb65ed-f890-41b2-8908-d0f299b6ba95 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:28.879996 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cb65ed-f890-41b2-8908-d0f299b6ba95 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:28.882170 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 169/325] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 04:58:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 18 04:58:28.888587 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8d3077-c0d3-421e-9ebd-085d7efb71d3 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:28.888911 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8d3077-c0d3-421e-9ebd-085d7efb71d3 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:28.889143 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8d3077-c0d3-421e-9ebd-085d7efb71d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:28.906727 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-6c8d3077-c0d3-421e-9ebd-085d7efb71d3 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 04:58:28.907918 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-6c8d3077-c0d3-421e-9ebd-085d7efb71d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 04:58:28.992732 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 170/326] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 04:58:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 18 04:58:29.000066 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28569619-a00f-425f-b577-5616c0372031 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:29.016084 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8d3077-c0d3-421e-9ebd-085d7efb71d3 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:29.019121 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:29.046268 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa53cfb2-9922-40e5-8f48-75caf4fe68d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:29.047071 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa53cfb2-9922-40e5-8f48-75caf4fe68d9 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:29.047542 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa53cfb2-9922-40e5-8f48-75caf4fe68d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:29.047963 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa53cfb2-9922-40e5-8f48-75caf4fe68d9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:29.060723 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa53cfb2-9922-40e5-8f48-75caf4fe68d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 18 04:58:29.069745 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application [None req-fa53cfb2-9922-40e5-8f48-75caf4fe68d9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 18 04:58:29.069745 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:58:29.069745 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:58:29.069745 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:58:29.069745 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:58:29.069745 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:58:29.069745 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:58:29.069745 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:58:29.069745 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:58:29.069745 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:58:29.069745 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:58:29.069745 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:58:29.069745 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 18 04:58:29.069745 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 18 04:58:29.069745 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 18 04:58:29.069745 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 18 04:58:29.070864 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 18 04:58:29.070864 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 18 04:58:29.070864 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 18 04:58:29.070864 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 18 04:58:29.070864 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:58:29.070864 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:58:29.070864 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:58:29.070864 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:58:29.070864 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:58:29.070864 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:58:29.070864 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:58:29.070864 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application with Lock( Jun 18 04:58:29.070864 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:58:29.070864 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:58:29.070864 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:58:29.070864 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:58:29.071851 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:58:29.071851 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:58:29.071851 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:58:29.071851 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:58:29.071851 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 18 04:58:29.071851 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 18 04:58:29.071851 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 18 04:58:29.071851 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 18 04:58:29.071851 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 18 04:58:29.071851 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 18 04:58:29.071851 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 18 04:58:29.071851 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 04:58:29.074328 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 157/327] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Jun 18 04:58:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:58:29.079664 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa53cfb2-9922-40e5-8f48-75caf4fe68d9 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:29.095522 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:29.149243 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8d3077-c0d3-421e-9ebd-085d7efb71d3 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:29.188476 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b271df-71ef-4efd-8637-27034b9515b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:29.189715 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b271df-71ef-4efd-8637-27034b9515b8 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:29.190144 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b271df-71ef-4efd-8637-27034b9515b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:29.190553 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b271df-71ef-4efd-8637-27034b9515b8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:29.202724 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa53cfb2-9922-40e5-8f48-75caf4fe68d9 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:29.204769 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b271df-71ef-4efd-8637-27034b9515b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 18 04:58:29.214741 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-36b271df-71ef-4efd-8637-27034b9515b8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 18 04:58:29.214741 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:58:29.214741 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:58:29.214741 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:58:29.214741 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:58:29.214741 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:58:29.214741 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:58:29.214741 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:58:29.214741 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:58:29.214741 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:58:29.214741 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:58:29.214741 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:58:29.214741 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 18 04:58:29.214741 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 18 04:58:29.214741 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 18 04:58:29.214741 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 18 04:58:29.215830 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 18 04:58:29.215830 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 18 04:58:29.215830 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 18 04:58:29.215830 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 18 04:58:29.215830 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:58:29.215830 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:58:29.215830 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:58:29.215830 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:58:29.215830 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:58:29.215830 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:58:29.215830 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:58:29.215830 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jun 18 04:58:29.215830 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:58:29.215830 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:58:29.215830 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:58:29.215830 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:58:29.216811 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:58:29.216811 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:58:29.216811 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:58:29.216811 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:58:29.216811 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 18 04:58:29.216811 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 18 04:58:29.216811 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 18 04:58:29.216811 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 18 04:58:29.216811 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 18 04:58:29.216811 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 18 04:58:29.216811 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 18 04:58:29.216811 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 04:58:29.219526 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 171/328] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Jun 18 04:58:29 2022] GET /identity/v3/roles/reader => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 18 04:58:29.226080 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b271df-71ef-4efd-8637-27034b9515b8 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:29.248542 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c9e747-e4b8-4ac0-aad2-859902449422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:29.249369 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c9e747-e4b8-4ac0-aad2-859902449422 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:29.249916 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c9e747-e4b8-4ac0-aad2-859902449422 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:29.250406 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c9e747-e4b8-4ac0-aad2-859902449422 None admin] PATH_INFO: `/v3/roles` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:29.251395 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c9e747-e4b8-4ac0-aad2-859902449422 None admin] RBAC: Adding query filter params (name=member) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:58:29.251971 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c9e747-e4b8-4ac0-aad2-859902449422 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:29.253642 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c9e747-e4b8-4ac0-aad2-859902449422 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:29.264428 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 158/329] 10.222.0.23 () {56 vars in 1101 bytes} [Sat Jun 18 04:58:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 18 04:58:29.270009 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c9e747-e4b8-4ac0-aad2-859902449422 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:29.313763 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:29.340571 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b271df-71ef-4efd-8637-27034b9515b8 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:29.364868 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:29.381472 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c9e747-e4b8-4ac0-aad2-859902449422 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:29.385400 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69258d36-f7a4-4105-bd42-9fad5370ea12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:29.390148 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69258d36-f7a4-4105-bd42-9fad5370ea12 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:29.390148 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69258d36-f7a4-4105-bd42-9fad5370ea12 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:29.390148 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69258d36-f7a4-4105-bd42-9fad5370ea12 None admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:29.390148 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69258d36-f7a4-4105-bd42-9fad5370ea12 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:58:29.390482 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69258d36-f7a4-4105-bd42-9fad5370ea12 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:29.392844 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69258d36-f7a4-4105-bd42-9fad5370ea12 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:29.405605 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 172/330] 10.222.0.23 () {56 vars in 1101 bytes} [Sat Jun 18 04:58:29 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 18 04:58:29.411101 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69258d36-f7a4-4105-bd42-9fad5370ea12 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:29.428404 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefee80e-a94e-422e-b9d8-18075bcb92ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:29.429728 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefee80e-a94e-422e-b9d8-18075bcb92ba None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:29.430212 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefee80e-a94e-422e-b9d8-18075bcb92ba None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:29.430785 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefee80e-a94e-422e-b9d8-18075bcb92ba None admin] PATH_INFO: `/v3/users/11fd155f3b5b4efd9abeb3d4952fe438` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:29.432509 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefee80e-a94e-422e-b9d8-18075bcb92ba None admin] RBAC: Authorizing `identity:get_user(user_id=11fd155f3b5b4efd9abeb3d4952fe438)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:29.434244 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefee80e-a94e-422e-b9d8-18075bcb92ba None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:29.436639 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 159/331] 10.222.0.23 () {56 vars in 1177 bytes} [Sat Jun 18 04:58:29 2022] GET /identity/v3/users/11fd155f3b5b4efd9abeb3d4952fe438 => generated 293 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 18 04:58:29.442892 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefee80e-a94e-422e-b9d8-18075bcb92ba None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:29.524361 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69258d36-f7a4-4105-bd42-9fad5370ea12 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:29.562202 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6898d93-8b4c-41a5-ac3c-924cc945ff2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:29.563088 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6898d93-8b4c-41a5-ac3c-924cc945ff2e None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:29.563376 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6898d93-8b4c-41a5-ac3c-924cc945ff2e None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:29.563659 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6898d93-8b4c-41a5-ac3c-924cc945ff2e None admin] PATH_INFO: `/v3/users/62d3bf53205f44f781ca83e32e58f523` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:29.564015 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefee80e-a94e-422e-b9d8-18075bcb92ba None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:29.565912 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6898d93-8b4c-41a5-ac3c-924cc945ff2e None admin] RBAC: Authorizing `identity:get_user(user_id=62d3bf53205f44f781ca83e32e58f523)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:29.567559 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6898d93-8b4c-41a5-ac3c-924cc945ff2e None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:29.573759 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 173/332] 10.222.0.23 () {56 vars in 1177 bytes} [Sat Jun 18 04:58:29 2022] GET /identity/v3/users/62d3bf53205f44f781ca83e32e58f523 => generated 293 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:29.575265 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6898d93-8b4c-41a5-ac3c-924cc945ff2e None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:29.592719 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:29.606251 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d23dd0-7bd4-4df2-9678-428a4ccf9295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:29.609183 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d23dd0-7bd4-4df2-9678-428a4ccf9295 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:29.609183 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d23dd0-7bd4-4df2-9678-428a4ccf9295 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:29.609183 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d23dd0-7bd4-4df2-9678-428a4ccf9295 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:29.609183 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d23dd0-7bd4-4df2-9678-428a4ccf9295 None admin] RBAC: Adding query filter params (role.id=cbf5c4c26898413d93e73b46b41046a3, user.id=11fd155f3b5b4efd9abeb3d4952fe438, scope.system=all) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:58:29.609499 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d23dd0-7bd4-4df2-9678-428a4ccf9295 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:29.611271 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d23dd0-7bd4-4df2-9678-428a4ccf9295 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:29.629689 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 160/333] 10.222.0.23 () {56 vars in 1308 bytes} [Sat Jun 18 04:58:29 2022] GET /identity/v3/role_assignments?user.id=11fd155f3b5b4efd9abeb3d4952fe438&scope.system=all&role.id=cbf5c4c26898413d93e73b46b41046a3 => generated 510 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:29.672898 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:29.708031 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6898d93-8b4c-41a5-ac3c-924cc945ff2e None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:29.753421 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1bc7c7-5293-4e22-aa7c-5ddb0b2a3f60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:29.754283 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1bc7c7-5293-4e22-aa7c-5ddb0b2a3f60 None admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:29.754571 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1bc7c7-5293-4e22-aa7c-5ddb0b2a3f60 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:29.754855 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1bc7c7-5293-4e22-aa7c-5ddb0b2a3f60 None admin] PATH_INFO: `/v3/system/users/62d3bf53205f44f781ca83e32e58f523/roles/80cac029fc6540658713c822fdd75949` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:29.757285 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1bc7c7-5293-4e22-aa7c-5ddb0b2a3f60 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=62d3bf53205f44f781ca83e32e58f523, role_id=80cac029fc6540658713c822fdd75949)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:29.758773 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1bc7c7-5293-4e22-aa7c-5ddb0b2a3f60 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:29.826219 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 174/334] 10.222.0.23 () {58 vars in 1334 bytes} [Sat Jun 18 04:58:29 2022] PUT /identity/v3/system/users/62d3bf53205f44f781ca83e32e58f523/roles/80cac029fc6540658713c822fdd75949 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 04:58:31.391016 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b090f7f-f4dd-4318-9cf8-870c45d7cca9 None None] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:31.391016 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b090f7f-f4dd-4318-9cf8-870c45d7cca9 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:31.391016 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b090f7f-f4dd-4318-9cf8-870c45d7cca9 None None] PATH_INFO: `/` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:31.392813 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 161/335] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 04:58:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 04:58:31.398117 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ba8525-4dad-405d-a5ab-1e5e2dcec388 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:31.398528 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ba8525-4dad-405d-a5ab-1e5e2dcec388 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:31.398696 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ba8525-4dad-405d-a5ab-1e5e2dcec388 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:31.422763 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-07ba8525-4dad-405d-a5ab-1e5e2dcec388 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 04:58:31.423972 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-07ba8525-4dad-405d-a5ab-1e5e2dcec388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 04:58:31.504366 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 175/336] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 04:58:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 18 04:58:31.527544 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10facb88-7d79-4ee0-8db6-3b1a23f3d69e None None] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:31.527844 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10facb88-7d79-4ee0-8db6-3b1a23f3d69e None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:31.528114 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10facb88-7d79-4ee0-8db6-3b1a23f3d69e None None] PATH_INFO: `/` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:31.529741 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 162/337] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 04:58:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 04:58:31.535164 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df2138e-efc8-4b0b-b65a-89f6e76909d2 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:31.535457 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df2138e-efc8-4b0b-b65a-89f6e76909d2 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:31.535739 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df2138e-efc8-4b0b-b65a-89f6e76909d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:31.554498 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10facb88-7d79-4ee0-8db6-3b1a23f3d69e None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:31.556981 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-7df2138e-efc8-4b0b-b65a-89f6e76909d2 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 04:58:31.561406 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-7df2138e-efc8-4b0b-b65a-89f6e76909d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 04:58:31.621575 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:31.715724 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 176/338] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 04:58:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 18 04:58:31.753363 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df2138e-efc8-4b0b-b65a-89f6e76909d2 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:31.819615 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10facb88-7d79-4ee0-8db6-3b1a23f3d69e None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:31.899499 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac918fc-6fcf-40bd-8d7d-3300b4e70eb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:31.902560 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac918fc-6fcf-40bd-8d7d-3300b4e70eb0 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:31.903065 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac918fc-6fcf-40bd-8d7d-3300b4e70eb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:31.906245 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac918fc-6fcf-40bd-8d7d-3300b4e70eb0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:31.925355 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac918fc-6fcf-40bd-8d7d-3300b4e70eb0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 18 04:58:31.933874 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application [None req-6ac918fc-6fcf-40bd-8d7d-3300b4e70eb0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 18 04:58:31.933874 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:58:31.933874 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:58:31.933874 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:58:31.933874 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:58:31.933874 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:58:31.933874 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:58:31.933874 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:58:31.933874 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:58:31.933874 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:58:31.933874 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:58:31.933874 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:58:31.933874 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 18 04:58:31.933874 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 18 04:58:31.933874 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 18 04:58:31.933874 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 18 04:58:31.934740 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 18 04:58:31.934740 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 18 04:58:31.934740 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 18 04:58:31.934740 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 18 04:58:31.934740 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:58:31.934740 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:58:31.934740 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:58:31.934740 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:58:31.934740 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:58:31.934740 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:58:31.934740 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:58:31.934740 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application with Lock( Jun 18 04:58:31.934740 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:58:31.934740 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:58:31.934740 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:58:31.934740 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:58:31.935508 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:58:31.935508 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:58:31.935508 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:58:31.935508 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:58:31.935508 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 18 04:58:31.935508 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 18 04:58:31.935508 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 18 04:58:31.935508 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 18 04:58:31.935508 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 18 04:58:31.935508 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 18 04:58:31.935508 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 18 04:58:31.935508 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 04:58:31.942881 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 163/339] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Jun 18 04:58:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 389 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:58:31.948870 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac918fc-6fcf-40bd-8d7d-3300b4e70eb0 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:31.966620 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df2138e-efc8-4b0b-b65a-89f6e76909d2 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:32.002167 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:32.038207 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d8c956-5669-407b-87d2-a150e9442f66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:32.039383 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d8c956-5669-407b-87d2-a150e9442f66 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:32.039859 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d8c956-5669-407b-87d2-a150e9442f66 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:32.040350 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d8c956-5669-407b-87d2-a150e9442f66 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:32.063517 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d8c956-5669-407b-87d2-a150e9442f66 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 18 04:58:32.073416 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-e6d8c956-5669-407b-87d2-a150e9442f66 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 18 04:58:32.073416 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:58:32.073416 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:58:32.073416 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:58:32.073416 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:58:32.073416 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:58:32.073416 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:58:32.073416 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:58:32.073416 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:58:32.073416 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:58:32.073416 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:58:32.073416 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:58:32.073416 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 18 04:58:32.073416 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 18 04:58:32.073416 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 18 04:58:32.073416 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 18 04:58:32.074325 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 18 04:58:32.074325 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 18 04:58:32.074325 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 18 04:58:32.074325 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 18 04:58:32.074325 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:58:32.074325 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:58:32.074325 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:58:32.074325 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:58:32.074325 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:58:32.074325 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:58:32.074325 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:58:32.074325 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jun 18 04:58:32.074325 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:58:32.074325 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:58:32.074325 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:58:32.074325 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:58:32.075116 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:58:32.075116 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:58:32.075116 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:58:32.075116 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:58:32.075116 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 18 04:58:32.075116 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 18 04:58:32.075116 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 18 04:58:32.075116 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 18 04:58:32.075116 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 18 04:58:32.075116 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 18 04:58:32.075116 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 18 04:58:32.075116 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 04:58:32.079813 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 177/340] 10.222.0.23 () {56 vars in 1114 bytes} [Sat Jun 18 04:58:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:58:32.086154 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d8c956-5669-407b-87d2-a150e9442f66 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:32.161522 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:32.171676 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac918fc-6fcf-40bd-8d7d-3300b4e70eb0 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:32.238692 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daa2cbf-7e4f-4882-bf10-9091f19aab31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:32.239287 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daa2cbf-7e4f-4882-bf10-9091f19aab31 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:32.239827 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daa2cbf-7e4f-4882-bf10-9091f19aab31 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:32.240104 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daa2cbf-7e4f-4882-bf10-9091f19aab31 None admin] PATH_INFO: `/v3/roles` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:32.241454 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8daa2cbf-7e4f-4882-bf10-9091f19aab31 None admin] RBAC: Adding query filter params (name=member) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:58:32.241973 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8daa2cbf-7e4f-4882-bf10-9091f19aab31 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:32.245251 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8daa2cbf-7e4f-4882-bf10-9091f19aab31 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:32.261017 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 164/341] 10.222.0.23 () {56 vars in 1101 bytes} [Sat Jun 18 04:58:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 18 04:58:32.272542 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d8c956-5669-407b-87d2-a150e9442f66 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:32.277877 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daa2cbf-7e4f-4882-bf10-9091f19aab31 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:32.330181 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e64454-0f96-48be-870d-efe457d61c13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:32.330856 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e64454-0f96-48be-870d-efe457d61c13 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:32.331138 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e64454-0f96-48be-870d-efe457d61c13 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:32.331412 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e64454-0f96-48be-870d-efe457d61c13 None admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:32.332164 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e64454-0f96-48be-870d-efe457d61c13 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:58:32.332477 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e64454-0f96-48be-870d-efe457d61c13 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:32.337899 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e64454-0f96-48be-870d-efe457d61c13 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:32.356460 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 178/342] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Jun 18 04:58:32 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:32.361896 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e64454-0f96-48be-870d-efe457d61c13 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:32.464305 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:32.479240 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daa2cbf-7e4f-4882-bf10-9091f19aab31 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:32.524400 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76cf265-9b63-47b6-b132-83a7270f9245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:32.525086 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76cf265-9b63-47b6-b132-83a7270f9245 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:32.525963 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76cf265-9b63-47b6-b132-83a7270f9245 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:32.526251 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76cf265-9b63-47b6-b132-83a7270f9245 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:32.580650 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:32.588359 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e64454-0f96-48be-870d-efe457d61c13 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:32.639230 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76cf265-9b63-47b6-b132-83a7270f9245 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:32.640698 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76cf265-9b63-47b6-b132-83a7270f9245 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:32.651441 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48936aba-76a8-48b0-b520-d17403b7281f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:32.652863 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48936aba-76a8-48b0-b520-d17403b7281f None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:32.653358 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48936aba-76a8-48b0-b520-d17403b7281f None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:32.653861 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48936aba-76a8-48b0-b520-d17403b7281f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:32.754768 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:32.767633 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 165/343] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Jun 18 04:58:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 490 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 18 04:58:32.767798 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48936aba-76a8-48b0-b520-d17403b7281f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:32.768962 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0662f59a-553f-4887-b7a2-961eaa57f917 None None] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:32.769113 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48936aba-76a8-48b0-b520-d17403b7281f None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:32.769245 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0662f59a-553f-4887-b7a2-961eaa57f917 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:32.769564 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0662f59a-553f-4887-b7a2-961eaa57f917 None None] PATH_INFO: `/` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:32.771187 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 166/344] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 04:58:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 04:58:32.773063 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0662f59a-553f-4887-b7a2-961eaa57f917 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:32.933691 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:32.939752 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 179/345] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Jun 18 04:58:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 579 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 18 04:58:32.941032 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056a5bf6-3b73-4074-bb37-377c50bbd2b7 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:32.941319 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056a5bf6-3b73-4074-bb37-377c50bbd2b7 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:32.941624 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056a5bf6-3b73-4074-bb37-377c50bbd2b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:32.946268 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0662f59a-553f-4887-b7a2-961eaa57f917 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:32.977330 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-056a5bf6-3b73-4074-bb37-377c50bbd2b7 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 04:58:32.978575 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-056a5bf6-3b73-4074-bb37-377c50bbd2b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 04:58:33.015969 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d45764-7280-482e-85e1-8ad98372c0e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:33.016834 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d45764-7280-482e-85e1-8ad98372c0e9 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:33.017113 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d45764-7280-482e-85e1-8ad98372c0e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:33.017404 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d45764-7280-482e-85e1-8ad98372c0e9 None admin] PATH_INFO: `/v3/projects/6706fdb556af472c8319dd2a13abe441` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:33.018600 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d45764-7280-482e-85e1-8ad98372c0e9 None admin] RBAC: Authorizing `identity:get_project(project_id=6706fdb556af472c8319dd2a13abe441)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:33.020021 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d45764-7280-482e-85e1-8ad98372c0e9 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:33.022869 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 167/346] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Jun 18 04:58:32 2022] GET /identity/v3/projects/6706fdb556af472c8319dd2a13abe441 => generated 298 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:33.023714 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d45764-7280-482e-85e1-8ad98372c0e9 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:33.072492 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 180/347] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 04:58:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 18 04:58:33.073851 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056a5bf6-3b73-4074-bb37-377c50bbd2b7 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:33.138011 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:33.207223 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d45764-7280-482e-85e1-8ad98372c0e9 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:33.211030 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056a5bf6-3b73-4074-bb37-377c50bbd2b7 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:33.241612 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:33.250153 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f298b45d-d6f8-45f1-baa3-f16923b6c36a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:33.254096 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f298b45d-d6f8-45f1-baa3-f16923b6c36a None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:33.254096 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f298b45d-d6f8-45f1-baa3-f16923b6c36a None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:33.254096 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f298b45d-d6f8-45f1-baa3-f16923b6c36a None admin] PATH_INFO: `/v3/groups/0edcd696603f45549001165885fb7a69` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:33.257705 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8627edd6-1620-4c80-8b3a-4e20be54c70c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:33.257975 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f298b45d-d6f8-45f1-baa3-f16923b6c36a None admin] RBAC: Authorizing `identity:get_group(group_id=0edcd696603f45549001165885fb7a69)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:33.259096 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8627edd6-1620-4c80-8b3a-4e20be54c70c None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:33.259445 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f298b45d-d6f8-45f1-baa3-f16923b6c36a None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:33.259792 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8627edd6-1620-4c80-8b3a-4e20be54c70c None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:33.260277 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8627edd6-1620-4c80-8b3a-4e20be54c70c None admin] PATH_INFO: `/v3/projects/6706fdb556af472c8319dd2a13abe441` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:33.261194 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 181/348] 10.222.0.23 () {56 vars in 1180 bytes} [Sat Jun 18 04:58:33 2022] GET /identity/v3/groups/0edcd696603f45549001165885fb7a69 => generated 226 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:33.262334 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8627edd6-1620-4c80-8b3a-4e20be54c70c None admin] RBAC: Authorizing `identity:get_project(project_id=6706fdb556af472c8319dd2a13abe441)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:33.263987 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8627edd6-1620-4c80-8b3a-4e20be54c70c None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:33.264577 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f22fed5-3c22-448a-82f2-1f4aa201e615 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:33.264863 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f22fed5-3c22-448a-82f2-1f4aa201e615 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:33.265139 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f22fed5-3c22-448a-82f2-1f4aa201e615 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:33.266971 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 168/349] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Jun 18 04:58:33 2022] GET /identity/v3/projects/6706fdb556af472c8319dd2a13abe441 => generated 298 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:33.267783 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 182/350] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 04:58:33 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 04:58:33.268256 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8627edd6-1620-4c80-8b3a-4e20be54c70c None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:33.268502 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f22fed5-3c22-448a-82f2-1f4aa201e615 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:33.402590 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8627edd6-1620-4c80-8b3a-4e20be54c70c None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:33.415099 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f22fed5-3c22-448a-82f2-1f4aa201e615 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:33.433265 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:33.446311 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065ecc5c-0e74-493f-84c4-fabe54a06e03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:33.451518 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065ecc5c-0e74-493f-84c4-fabe54a06e03 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:33.451518 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065ecc5c-0e74-493f-84c4-fabe54a06e03 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:33.451518 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065ecc5c-0e74-493f-84c4-fabe54a06e03 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:33.456985 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bc7532-49b5-4105-8e5b-7d1a9ae46bd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:33.459132 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bc7532-49b5-4105-8e5b-7d1a9ae46bd6 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:33.459361 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065ecc5c-0e74-493f-84c4-fabe54a06e03 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 18 04:58:33.460058 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bc7532-49b5-4105-8e5b-7d1a9ae46bd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:33.460541 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bc7532-49b5-4105-8e5b-7d1a9ae46bd6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:33.461654 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bc7532-49b5-4105-8e5b-7d1a9ae46bd6 None admin] RBAC: Adding query filter params (group.id=0edcd696603f45549001165885fb7a69, role.id=cbf5c4c26898413d93e73b46b41046a3, scope.project.id=6706fdb556af472c8319dd2a13abe441) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:58:33.462169 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bc7532-49b5-4105-8e5b-7d1a9ae46bd6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:33.464790 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bc7532-49b5-4105-8e5b-7d1a9ae46bd6 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:33.467685 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application [None req-065ecc5c-0e74-493f-84c4-fabe54a06e03 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 18 04:58:33.467685 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:58:33.467685 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:58:33.467685 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:58:33.467685 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:58:33.467685 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:58:33.467685 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:58:33.467685 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:58:33.467685 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:58:33.467685 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:58:33.467685 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:58:33.467685 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:58:33.467685 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 18 04:58:33.467685 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 18 04:58:33.467685 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 18 04:58:33.467685 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 18 04:58:33.469258 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 18 04:58:33.469258 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 18 04:58:33.469258 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 18 04:58:33.469258 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 18 04:58:33.469258 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:58:33.469258 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:58:33.469258 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:58:33.469258 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:58:33.469258 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:58:33.469258 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:58:33.469258 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:58:33.469258 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application with Lock( Jun 18 04:58:33.469258 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:58:33.469258 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:58:33.469258 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:58:33.469258 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:58:33.470911 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:58:33.470911 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:58:33.470911 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:58:33.470911 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:58:33.470911 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 18 04:58:33.470911 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 18 04:58:33.470911 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 18 04:58:33.470911 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 18 04:58:33.470911 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 18 04:58:33.470911 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 18 04:58:33.470911 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 18 04:58:33.470911 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 04:58:33.472972 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 169/351] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Jun 18 04:58:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:58:33.474033 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065ecc5c-0e74-493f-84c4-fabe54a06e03 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:33.479548 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 183/352] 10.222.0.23 () {56 vars in 1376 bytes} [Sat Jun 18 04:58:33 2022] GET /identity/v3/role_assignments?group.id=0edcd696603f45549001165885fb7a69&scope.project.id=6706fdb556af472c8319dd2a13abe441&role.id=cbf5c4c26898413d93e73b46b41046a3 => generated 260 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:33.481081 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3a6456-3228-43e0-b9ae-601f6fc1b02c None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:33.481494 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3a6456-3228-43e0-b9ae-601f6fc1b02c None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:33.481869 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3a6456-3228-43e0-b9ae-601f6fc1b02c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:33.501693 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-ba3a6456-3228-43e0-b9ae-601f6fc1b02c None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 04:58:33.504027 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-ba3a6456-3228-43e0-b9ae-601f6fc1b02c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 04:58:33.566675 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:33.586552 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 184/353] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 04:58:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 18 04:58:33.587633 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3a6456-3228-43e0-b9ae-601f6fc1b02c None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:33.603568 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065ecc5c-0e74-493f-84c4-fabe54a06e03 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:33.645814 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418bc378-8814-45d7-90ed-6dd6bb101b7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:33.647092 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418bc378-8814-45d7-90ed-6dd6bb101b7c None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:33.647586 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418bc378-8814-45d7-90ed-6dd6bb101b7c None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:33.648071 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418bc378-8814-45d7-90ed-6dd6bb101b7c None admin] PATH_INFO: `/v3/groups/0edcd696603f45549001165885fb7a69` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:33.649855 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418bc378-8814-45d7-90ed-6dd6bb101b7c None admin] RBAC: Authorizing `identity:get_group(group_id=0edcd696603f45549001165885fb7a69)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:33.651606 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418bc378-8814-45d7-90ed-6dd6bb101b7c None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:33.655071 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 170/354] 10.222.0.23 () {56 vars in 1180 bytes} [Sat Jun 18 04:58:33 2022] GET /identity/v3/groups/0edcd696603f45549001165885fb7a69 => generated 226 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:33.656949 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552f0096-a4b0-4356-8609-423347e7c67b None None] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:33.657429 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552f0096-a4b0-4356-8609-423347e7c67b None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:33.657934 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552f0096-a4b0-4356-8609-423347e7c67b None None] PATH_INFO: `/` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:33.659997 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 171/355] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 04:58:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 04:58:33.660953 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552f0096-a4b0-4356-8609-423347e7c67b None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:33.703818 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3a6456-3228-43e0-b9ae-601f6fc1b02c None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:33.762983 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:33.790176 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9816989-64a3-46a6-a34b-985dd2c714b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:33.790859 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9816989-64a3-46a6-a34b-985dd2c714b9 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:33.791142 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9816989-64a3-46a6-a34b-985dd2c714b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:33.791422 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9816989-64a3-46a6-a34b-985dd2c714b9 None admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:33.792176 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9816989-64a3-46a6-a34b-985dd2c714b9 None admin] RBAC: Adding query filter params (name=member) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:58:33.792494 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9816989-64a3-46a6-a34b-985dd2c714b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:33.793995 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9816989-64a3-46a6-a34b-985dd2c714b9 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:33.806672 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 185/356] 10.222.0.23 () {56 vars in 1101 bytes} [Sat Jun 18 04:58:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:33.810175 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9816989-64a3-46a6-a34b-985dd2c714b9 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:33.850594 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552f0096-a4b0-4356-8609-423347e7c67b None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:33.877546 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:33.898946 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e0410c-1f95-4646-840e-1a1fb4ca43d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:33.899738 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e0410c-1f95-4646-840e-1a1fb4ca43d6 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:33.900020 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e0410c-1f95-4646-840e-1a1fb4ca43d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:33.900296 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e0410c-1f95-4646-840e-1a1fb4ca43d6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:33.911574 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e0410c-1f95-4646-840e-1a1fb4ca43d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 18 04:58:33.922728 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application [None req-60e0410c-1f95-4646-840e-1a1fb4ca43d6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 18 04:58:33.922728 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:58:33.922728 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:58:33.922728 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:58:33.922728 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:58:33.922728 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:58:33.922728 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:58:33.922728 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:58:33.922728 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:58:33.922728 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:58:33.922728 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:58:33.922728 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:58:33.922728 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 18 04:58:33.922728 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 18 04:58:33.922728 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 18 04:58:33.922728 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 18 04:58:33.923633 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 18 04:58:33.923633 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 18 04:58:33.923633 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 18 04:58:33.923633 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 18 04:58:33.923633 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:58:33.923633 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:58:33.923633 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:58:33.923633 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:58:33.923633 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:58:33.923633 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:58:33.923633 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:58:33.923633 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application with Lock( Jun 18 04:58:33.923633 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:58:33.923633 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:58:33.923633 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:58:33.923633 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:58:33.924430 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:58:33.924430 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:58:33.924430 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:58:33.924430 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:58:33.924430 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 18 04:58:33.924430 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 18 04:58:33.924430 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 18 04:58:33.924430 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 18 04:58:33.924430 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 18 04:58:33.924430 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 18 04:58:33.924430 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 18 04:58:33.924430 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 04:58:33.924430 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 172/357] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Jun 18 04:58:33 2022] GET /identity/v3/roles/reader => generated 84 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:58:33.925736 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85704ba6-3a9f-4e92-a9f4-adfbe3ef4f65 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:33.925736 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85704ba6-3a9f-4e92-a9f4-adfbe3ef4f65 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:33.925736 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85704ba6-3a9f-4e92-a9f4-adfbe3ef4f65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:33.946694 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-85704ba6-3a9f-4e92-a9f4-adfbe3ef4f65 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 04:58:33.948251 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-85704ba6-3a9f-4e92-a9f4-adfbe3ef4f65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 04:58:33.954181 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9816989-64a3-46a6-a34b-985dd2c714b9 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:33.999175 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd25baa-b7c1-4ca3-bdc5-e6f713dbacca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:34.000201 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd25baa-b7c1-4ca3-bdc5-e6f713dbacca None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:34.001153 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd25baa-b7c1-4ca3-bdc5-e6f713dbacca None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:34.001678 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd25baa-b7c1-4ca3-bdc5-e6f713dbacca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:34.003142 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd25baa-b7c1-4ca3-bdc5-e6f713dbacca None admin] RBAC: Adding query filter params (group.id=0edcd696603f45549001165885fb7a69, role.id=0089a3aad83c4ae08d8e9d864957285b, scope.project.id=6706fdb556af472c8319dd2a13abe441) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:58:34.003668 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd25baa-b7c1-4ca3-bdc5-e6f713dbacca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:34.006493 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd25baa-b7c1-4ca3-bdc5-e6f713dbacca None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:34.012873 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 186/358] 10.222.0.23 () {56 vars in 1376 bytes} [Sat Jun 18 04:58:33 2022] GET /identity/v3/role_assignments?group.id=0edcd696603f45549001165885fb7a69&scope.project.id=6706fdb556af472c8319dd2a13abe441&role.id=0089a3aad83c4ae08d8e9d864957285b => generated 260 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:34.014432 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ea7192-5e06-41a6-9f02-0838029bb011 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:34.014432 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ea7192-5e06-41a6-9f02-0838029bb011 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:34.014616 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ea7192-5e06-41a6-9f02-0838029bb011 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:34.015766 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 187/359] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 04:58:34 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 04:58:34.016434 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ea7192-5e06-41a6-9f02-0838029bb011 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:34.041499 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 173/360] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 04:58:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 18 04:58:34.042481 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85704ba6-3a9f-4e92-a9f4-adfbe3ef4f65 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:34.107558 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:34.195536 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ea7192-5e06-41a6-9f02-0838029bb011 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:34.216606 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:34.217333 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85704ba6-3a9f-4e92-a9f4-adfbe3ef4f65 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:34.248228 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c71d92-af26-4928-b34d-7b8680f1b0d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:34.249255 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c71d92-af26-4928-b34d-7b8680f1b0d2 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:34.249982 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c71d92-af26-4928-b34d-7b8680f1b0d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:34.250437 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c71d92-af26-4928-b34d-7b8680f1b0d2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:34.265194 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5c7584-749f-44ca-860e-378cb8d3e911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:34.265828 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5c7584-749f-44ca-860e-378cb8d3e911 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:34.266113 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5c7584-749f-44ca-860e-378cb8d3e911 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:34.266384 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5c7584-749f-44ca-860e-378cb8d3e911 None admin] PATH_INFO: `/v3/roles` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:34.267406 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5c7584-749f-44ca-860e-378cb8d3e911 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:58:34.267406 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5c7584-749f-44ca-860e-378cb8d3e911 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:34.268767 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5c7584-749f-44ca-860e-378cb8d3e911 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:34.279061 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 174/361] 10.222.0.23 () {56 vars in 1101 bytes} [Sat Jun 18 04:58:34 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:34.281889 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb8c46f-9388-4326-89aa-841a5f7a2e04 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:34.282181 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb8c46f-9388-4326-89aa-841a5f7a2e04 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:34.282467 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb8c46f-9388-4326-89aa-841a5f7a2e04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:34.310714 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-efb8c46f-9388-4326-89aa-841a5f7a2e04 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 04:58:34.312031 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-efb8c46f-9388-4326-89aa-841a5f7a2e04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 04:58:34.339061 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c71d92-af26-4928-b34d-7b8680f1b0d2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:34.340768 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c71d92-af26-4928-b34d-7b8680f1b0d2 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:34.408822 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 175/362] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 04:58:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 18 04:58:34.411793 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb8c46f-9388-4326-89aa-841a5f7a2e04 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:34.457237 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:34.479416 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 188/363] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Jun 18 04:58:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 463 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 18 04:58:34.481257 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c71d92-af26-4928-b34d-7b8680f1b0d2 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:34.509469 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:34.563021 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb8c46f-9388-4326-89aa-841a5f7a2e04 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:34.604591 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2075c6a-35a8-45f2-898c-594ce8a9addc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:34.605430 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2075c6a-35a8-45f2-898c-594ce8a9addc None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:34.605696 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2075c6a-35a8-45f2-898c-594ce8a9addc None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:34.605967 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2075c6a-35a8-45f2-898c-594ce8a9addc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:34.625143 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2075c6a-35a8-45f2-898c-594ce8a9addc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 18 04:58:34.635473 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application [None req-e2075c6a-35a8-45f2-898c-594ce8a9addc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 18 04:58:34.635473 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:58:34.635473 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:58:34.635473 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:58:34.635473 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:58:34.635473 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:58:34.635473 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:58:34.635473 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:58:34.635473 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:58:34.635473 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:58:34.635473 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:58:34.635473 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:58:34.635473 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 18 04:58:34.635473 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 18 04:58:34.635473 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 18 04:58:34.635473 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 18 04:58:34.636402 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 18 04:58:34.636402 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 18 04:58:34.636402 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 18 04:58:34.636402 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 18 04:58:34.636402 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:58:34.636402 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:58:34.636402 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:58:34.636402 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:58:34.636402 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:58:34.636402 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:58:34.636402 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:58:34.636402 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application with Lock( Jun 18 04:58:34.636402 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:58:34.636402 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:58:34.636402 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:58:34.636402 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:58:34.637195 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:58:34.637195 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:58:34.637195 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:58:34.637195 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:58:34.637195 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 18 04:58:34.637195 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 18 04:58:34.637195 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 18 04:58:34.637195 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 18 04:58:34.637195 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 18 04:58:34.637195 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 18 04:58:34.637195 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 18 04:58:34.637195 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 04:58:34.637841 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c71d92-af26-4928-b34d-7b8680f1b0d2 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:34.638739 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 176/364] 10.222.0.23 () {56 vars in 1114 bytes} [Sat Jun 18 04:58:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:58:34.640170 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2075c6a-35a8-45f2-898c-594ce8a9addc None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:34.693170 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5347fa8-2a3d-430e-b78b-1c2d8498c873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:34.697993 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5347fa8-2a3d-430e-b78b-1c2d8498c873 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:34.697993 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5347fa8-2a3d-430e-b78b-1c2d8498c873 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:34.697993 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5347fa8-2a3d-430e-b78b-1c2d8498c873 None admin] PATH_INFO: `/v3/users/62d3bf53205f44f781ca83e32e58f523` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:34.697993 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5347fa8-2a3d-430e-b78b-1c2d8498c873 None admin] RBAC: Authorizing `identity:get_user(user_id=62d3bf53205f44f781ca83e32e58f523)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:34.697993 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5347fa8-2a3d-430e-b78b-1c2d8498c873 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:34.700183 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 189/365] 10.222.0.23 () {56 vars in 1177 bytes} [Sat Jun 18 04:58:34 2022] GET /identity/v3/users/62d3bf53205f44f781ca83e32e58f523 => generated 293 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:34.701218 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5347fa8-2a3d-430e-b78b-1c2d8498c873 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:34.759737 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:34.768799 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2075c6a-35a8-45f2-898c-594ce8a9addc None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:34.804498 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:34.804919 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b760d9-0c05-4a96-a71d-21dd9e337f68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:34.805889 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b760d9-0c05-4a96-a71d-21dd9e337f68 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:34.806464 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b760d9-0c05-4a96-a71d-21dd9e337f68 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:34.806731 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b760d9-0c05-4a96-a71d-21dd9e337f68 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:34.819545 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b760d9-0c05-4a96-a71d-21dd9e337f68 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 18 04:58:34.825938 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application [None req-a5b760d9-0c05-4a96-a71d-21dd9e337f68 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 18 04:58:34.825938 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:58:34.825938 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:58:34.825938 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:58:34.825938 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:58:34.825938 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:58:34.825938 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:58:34.825938 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:58:34.825938 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:58:34.825938 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:58:34.825938 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:58:34.825938 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:58:34.825938 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 18 04:58:34.825938 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 18 04:58:34.825938 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 18 04:58:34.825938 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 18 04:58:34.826977 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 18 04:58:34.826977 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 18 04:58:34.826977 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 18 04:58:34.826977 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 18 04:58:34.826977 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:58:34.826977 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:58:34.826977 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:58:34.826977 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:58:34.826977 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:58:34.826977 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:58:34.826977 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:58:34.826977 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application with Lock( Jun 18 04:58:34.826977 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:58:34.826977 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:58:34.826977 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:58:34.826977 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:58:34.828442 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:58:34.828442 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:58:34.828442 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:58:34.828442 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:58:34.828442 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 18 04:58:34.828442 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 18 04:58:34.828442 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 18 04:58:34.828442 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 18 04:58:34.828442 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 18 04:58:34.828442 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 18 04:58:34.828442 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 18 04:58:34.828442 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 04:58:34.828442 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 177/366] 10.222.0.23 () {56 vars in 1096 bytes} [Sat Jun 18 04:58:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:58:34.828442 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b760d9-0c05-4a96-a71d-21dd9e337f68 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:34.830820 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5347fa8-2a3d-430e-b78b-1c2d8498c873 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:34.870307 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6a0d52-54dc-473d-938e-7fb09ced892d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:34.871551 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6a0d52-54dc-473d-938e-7fb09ced892d None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:34.872034 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6a0d52-54dc-473d-938e-7fb09ced892d None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:34.872503 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6a0d52-54dc-473d-938e-7fb09ced892d None admin] PATH_INFO: `/v3/projects/41cb6d0c536f4c4494fdabc03643aef0` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:34.873951 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6a0d52-54dc-473d-938e-7fb09ced892d None admin] RBAC: Authorizing `identity:get_project(project_id=41cb6d0c536f4c4494fdabc03643aef0)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:34.875619 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6a0d52-54dc-473d-938e-7fb09ced892d None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:34.877789 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 190/367] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Jun 18 04:58:34 2022] GET /identity/v3/projects/41cb6d0c536f4c4494fdabc03643aef0 => generated 302 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:34.878877 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6a0d52-54dc-473d-938e-7fb09ced892d None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:34.942290 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b760d9-0c05-4a96-a71d-21dd9e337f68 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:34.979301 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec17a72c-d144-4806-b1b4-5f6dc77961e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:34.979838 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec17a72c-d144-4806-b1b4-5f6dc77961e0 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:34.980127 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec17a72c-d144-4806-b1b4-5f6dc77961e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:34.980407 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec17a72c-d144-4806-b1b4-5f6dc77961e0 None admin] PATH_INFO: `/v3/roles` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:34.981172 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec17a72c-d144-4806-b1b4-5f6dc77961e0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:58:34.981490 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec17a72c-d144-4806-b1b4-5f6dc77961e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:34.982886 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec17a72c-d144-4806-b1b4-5f6dc77961e0 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:34.986685 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6a0d52-54dc-473d-938e-7fb09ced892d None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:34.992274 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 178/368] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Jun 18 04:58:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:34.993168 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec17a72c-d144-4806-b1b4-5f6dc77961e0 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:35.011651 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:35.026910 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceee5058-7f7b-4ccb-b21e-175480b50680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:35.028210 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceee5058-7f7b-4ccb-b21e-175480b50680 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:35.028796 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceee5058-7f7b-4ccb-b21e-175480b50680 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:35.029347 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceee5058-7f7b-4ccb-b21e-175480b50680 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:35.030469 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceee5058-7f7b-4ccb-b21e-175480b50680 None admin] RBAC: Adding query filter params (role.id=80cac029fc6540658713c822fdd75949, user.id=62d3bf53205f44f781ca83e32e58f523, scope.system=all) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:58:35.031087 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceee5058-7f7b-4ccb-b21e-175480b50680 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:35.032902 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceee5058-7f7b-4ccb-b21e-175480b50680 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:35.047938 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 191/369] 10.222.0.23 () {56 vars in 1308 bytes} [Sat Jun 18 04:58:34 2022] GET /identity/v3/role_assignments?user.id=62d3bf53205f44f781ca83e32e58f523&scope.system=all&role.id=80cac029fc6540658713c822fdd75949 => generated 510 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:35.049412 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceee5058-7f7b-4ccb-b21e-175480b50680 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:35.061243 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:35.111151 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec17a72c-d144-4806-b1b4-5f6dc77961e0 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:35.149562 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aef041a-1795-4659-b2d8-19f429b81e04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:35.150244 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aef041a-1795-4659-b2d8-19f429b81e04 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:35.150528 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aef041a-1795-4659-b2d8-19f429b81e04 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:35.150802 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aef041a-1795-4659-b2d8-19f429b81e04 None admin] PATH_INFO: `/v3/roles` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:35.151560 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aef041a-1795-4659-b2d8-19f429b81e04 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:58:35.151871 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aef041a-1795-4659-b2d8-19f429b81e04 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:35.153315 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aef041a-1795-4659-b2d8-19f429b81e04 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:35.163518 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 179/370] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Jun 18 04:58:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:35.164235 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aef041a-1795-4659-b2d8-19f429b81e04 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:35.169417 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceee5058-7f7b-4ccb-b21e-175480b50680 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:35.207169 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5636927d-380d-48a3-a739-8f75ffe1f408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:35.208437 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5636927d-380d-48a3-a739-8f75ffe1f408 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:35.208985 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5636927d-380d-48a3-a739-8f75ffe1f408 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:35.209536 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5636927d-380d-48a3-a739-8f75ffe1f408 None admin] PATH_INFO: `/v3/groups/0edcd696603f45549001165885fb7a69` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:35.211631 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5636927d-380d-48a3-a739-8f75ffe1f408 None admin] RBAC: Authorizing `identity:get_group(group_id=0edcd696603f45549001165885fb7a69)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:35.213374 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5636927d-380d-48a3-a739-8f75ffe1f408 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:35.215522 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 192/371] 10.222.0.23 () {56 vars in 1180 bytes} [Sat Jun 18 04:58:35 2022] GET /identity/v3/groups/0edcd696603f45549001165885fb7a69 => generated 226 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:35.216753 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5636927d-380d-48a3-a739-8f75ffe1f408 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:35.277119 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aef041a-1795-4659-b2d8-19f429b81e04 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:35.314418 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f63b4d9-f98d-4ff1-9e97-371e351b4541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:35.315153 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f63b4d9-f98d-4ff1-9e97-371e351b4541 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:35.317353 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f63b4d9-f98d-4ff1-9e97-371e351b4541 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:35.317671 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:35.318504 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f63b4d9-f98d-4ff1-9e97-371e351b4541 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:35.329123 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5636927d-380d-48a3-a739-8f75ffe1f408 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:35.355677 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:35.371032 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c01f0b-23d0-454e-951e-083ba12a316b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:35.371032 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c01f0b-23d0-454e-951e-083ba12a316b None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:35.371032 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c01f0b-23d0-454e-951e-083ba12a316b None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:35.371728 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c01f0b-23d0-454e-951e-083ba12a316b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:35.394795 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f63b4d9-f98d-4ff1-9e97-371e351b4541 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:35.396965 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f63b4d9-f98d-4ff1-9e97-371e351b4541 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:35.447527 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c01f0b-23d0-454e-951e-083ba12a316b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:35.449376 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c01f0b-23d0-454e-951e-083ba12a316b None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:35.481264 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:35.508650 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 180/372] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Jun 18 04:58:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 18 04:58:35.509673 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f63b4d9-f98d-4ff1-9e97-371e351b4541 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:35.530895 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:35.559298 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 193/373] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Jun 18 04:58:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 18 04:58:35.560580 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c01f0b-23d0-454e-951e-083ba12a316b None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:35.616999 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f63b4d9-f98d-4ff1-9e97-371e351b4541 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:35.657600 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2541f6-d3ed-4efa-9b3a-398a0752d530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:35.658232 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2541f6-d3ed-4efa-9b3a-398a0752d530 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:35.658522 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2541f6-d3ed-4efa-9b3a-398a0752d530 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:35.658801 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2541f6-d3ed-4efa-9b3a-398a0752d530 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:35.659644 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2541f6-d3ed-4efa-9b3a-398a0752d530 None admin] RBAC: Adding query filter params (group.id=0edcd696603f45549001165885fb7a69, role.id=cbf5c4c26898413d93e73b46b41046a3, scope.project.id=41cb6d0c536f4c4494fdabc03643aef0) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:58:35.659960 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2541f6-d3ed-4efa-9b3a-398a0752d530 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:35.661456 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2541f6-d3ed-4efa-9b3a-398a0752d530 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:35.668410 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 181/374] 10.222.0.23 () {56 vars in 1376 bytes} [Sat Jun 18 04:58:35 2022] GET /identity/v3/role_assignments?group.id=0edcd696603f45549001165885fb7a69&scope.project.id=41cb6d0c536f4c4494fdabc03643aef0&role.id=cbf5c4c26898413d93e73b46b41046a3 => generated 260 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:35.669232 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2541f6-d3ed-4efa-9b3a-398a0752d530 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:35.675031 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c01f0b-23d0-454e-951e-083ba12a316b None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:35.714956 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a913c5-182b-4cb4-b907-d28b4324b85c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:35.716205 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a913c5-182b-4cb4-b907-d28b4324b85c None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:35.716695 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a913c5-182b-4cb4-b907-d28b4324b85c None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:35.717123 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:35.717421 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a913c5-182b-4cb4-b907-d28b4324b85c None admin] PATH_INFO: `/v3/projects/41cb6d0c536f4c4494fdabc03643aef0` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:35.718852 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a913c5-182b-4cb4-b907-d28b4324b85c None admin] RBAC: Authorizing `identity:get_project(project_id=41cb6d0c536f4c4494fdabc03643aef0)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:35.720514 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a913c5-182b-4cb4-b907-d28b4324b85c None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:35.722600 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 194/375] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Jun 18 04:58:35 2022] GET /identity/v3/projects/41cb6d0c536f4c4494fdabc03643aef0 => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:35.732930 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a913c5-182b-4cb4-b907-d28b4324b85c None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:35.788243 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:35.795944 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2541f6-d3ed-4efa-9b3a-398a0752d530 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:35.875031 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a913c5-182b-4cb4-b907-d28b4324b85c None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:35.881402 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3415fc32-9c95-4024-b831-f403f0eb4b89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:35.882295 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3415fc32-9c95-4024-b831-f403f0eb4b89 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:35.882580 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3415fc32-9c95-4024-b831-f403f0eb4b89 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:35.882858 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3415fc32-9c95-4024-b831-f403f0eb4b89 None admin] PATH_INFO: `/v3/projects/d587d7347f6f4de2882df28dc6b6a574` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:35.886066 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3415fc32-9c95-4024-b831-f403f0eb4b89 None admin] RBAC: Authorizing `identity:get_project(project_id=d587d7347f6f4de2882df28dc6b6a574)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:35.887523 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3415fc32-9c95-4024-b831-f403f0eb4b89 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:35.891524 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 182/376] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Jun 18 04:58:35 2022] GET /identity/v3/projects/d587d7347f6f4de2882df28dc6b6a574 => generated 344 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:35.895996 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3415fc32-9c95-4024-b831-f403f0eb4b89 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:35.921240 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ea0d27-9651-40c9-bf05-8793424c9c47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:35.922115 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ea0d27-9651-40c9-bf05-8793424c9c47 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:35.922406 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ea0d27-9651-40c9-bf05-8793424c9c47 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:35.922683 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ea0d27-9651-40c9-bf05-8793424c9c47 None admin] PATH_INFO: `/v3/groups/0edcd696603f45549001165885fb7a69` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:35.923895 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ea0d27-9651-40c9-bf05-8793424c9c47 None admin] RBAC: Authorizing `identity:get_group(group_id=0edcd696603f45549001165885fb7a69)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:35.925373 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ea0d27-9651-40c9-bf05-8793424c9c47 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:35.927378 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 195/377] 10.222.0.23 () {56 vars in 1180 bytes} [Sat Jun 18 04:58:35 2022] GET /identity/v3/groups/0edcd696603f45549001165885fb7a69 => generated 226 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:35.933862 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ea0d27-9651-40c9-bf05-8793424c9c47 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:36.045806 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3415fc32-9c95-4024-b831-f403f0eb4b89 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:36.057552 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:36.078208 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ea0d27-9651-40c9-bf05-8793424c9c47 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:36.089726 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f15463-6556-49e2-afae-c2c9928b7fe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:36.090588 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f15463-6556-49e2-afae-c2c9928b7fe8 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:36.090867 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f15463-6556-49e2-afae-c2c9928b7fe8 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:36.091136 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f15463-6556-49e2-afae-c2c9928b7fe8 None admin] PATH_INFO: `/v3/groups/6e930b1dfac546d6a830d4cc1808c375` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:36.091246 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:36.103187 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f15463-6556-49e2-afae-c2c9928b7fe8 None admin] RBAC: Authorizing `identity:get_group(group_id=6e930b1dfac546d6a830d4cc1808c375)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:36.104768 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f15463-6556-49e2-afae-c2c9928b7fe8 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:36.107587 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 183/378] 10.222.0.23 () {56 vars in 1180 bytes} [Sat Jun 18 04:58:35 2022] GET /identity/v3/groups/6e930b1dfac546d6a830d4cc1808c375 => generated 229 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 18 04:58:36.109012 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ad480e-a761-4e17-8cd2-4c1bf5af2613 None None] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:36.109299 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ad480e-a761-4e17-8cd2-4c1bf5af2613 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:36.109615 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ad480e-a761-4e17-8cd2-4c1bf5af2613 None None] PATH_INFO: `/` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:36.111100 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 184/379] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 04:58:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 04:58:36.116743 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ad480e-a761-4e17-8cd2-4c1bf5af2613 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:36.126482 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db41f85-aa47-4278-b0f9-7506e8ca5574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:36.127638 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db41f85-aa47-4278-b0f9-7506e8ca5574 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:36.127930 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db41f85-aa47-4278-b0f9-7506e8ca5574 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:36.128214 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db41f85-aa47-4278-b0f9-7506e8ca5574 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:36.129520 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db41f85-aa47-4278-b0f9-7506e8ca5574 None admin] RBAC: Adding query filter params (group.id=0edcd696603f45549001165885fb7a69, role.id=0089a3aad83c4ae08d8e9d864957285b, scope.project.id=41cb6d0c536f4c4494fdabc03643aef0) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:58:36.130264 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db41f85-aa47-4278-b0f9-7506e8ca5574 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:36.132795 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db41f85-aa47-4278-b0f9-7506e8ca5574 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:36.142023 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 196/380] 10.222.0.23 () {56 vars in 1376 bytes} [Sat Jun 18 04:58:35 2022] GET /identity/v3/role_assignments?group.id=0edcd696603f45549001165885fb7a69&scope.project.id=41cb6d0c536f4c4494fdabc03643aef0&role.id=0089a3aad83c4ae08d8e9d864957285b => generated 260 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 18 04:58:36.143305 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb465750-a4e6-49d0-9e5e-eda266ad80e2 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:36.144054 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb465750-a4e6-49d0-9e5e-eda266ad80e2 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:36.144348 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb465750-a4e6-49d0-9e5e-eda266ad80e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:36.166030 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-fb465750-a4e6-49d0-9e5e-eda266ad80e2 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 04:58:36.167270 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-fb465750-a4e6-49d0-9e5e-eda266ad80e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 04:58:36.310224 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 197/381] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 04:58:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 18 04:58:36.318496 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ad480e-a761-4e17-8cd2-4c1bf5af2613 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:36.344253 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb465750-a4e6-49d0-9e5e-eda266ad80e2 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:36.370166 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf3d1b4-442c-4a22-999c-2db6a5c31f28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:36.370832 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf3d1b4-442c-4a22-999c-2db6a5c31f28 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:36.371117 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf3d1b4-442c-4a22-999c-2db6a5c31f28 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:36.371394 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf3d1b4-442c-4a22-999c-2db6a5c31f28 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:36.372244 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf3d1b4-442c-4a22-999c-2db6a5c31f28 None admin] RBAC: Adding query filter params (group.id=6e930b1dfac546d6a830d4cc1808c375, role.id=21c65f2c51bd4dd283d13507d2714467, scope.project.id=d587d7347f6f4de2882df28dc6b6a574) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:58:36.372564 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf3d1b4-442c-4a22-999c-2db6a5c31f28 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:36.379930 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf3d1b4-442c-4a22-999c-2db6a5c31f28 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:36.392205 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 185/382] 10.222.0.23 () {56 vars in 1376 bytes} [Sat Jun 18 04:58:36 2022] GET /identity/v3/role_assignments?group.id=6e930b1dfac546d6a830d4cc1808c375&scope.project.id=d587d7347f6f4de2882df28dc6b6a574&role.id=21c65f2c51bd4dd283d13507d2714467 => generated 260 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:36.496137 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb465750-a4e6-49d0-9e5e-eda266ad80e2 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:36.498937 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:36.536887 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffc4774-08f1-4d61-b4ef-5ece546ec15d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:36.539193 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffc4774-08f1-4d61-b4ef-5ece546ec15d None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:36.539746 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffc4774-08f1-4d61-b4ef-5ece546ec15d None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:36.540219 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffc4774-08f1-4d61-b4ef-5ece546ec15d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:36.559368 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffc4774-08f1-4d61-b4ef-5ece546ec15d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 18 04:58:36.567781 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-0ffc4774-08f1-4d61-b4ef-5ece546ec15d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 18 04:58:36.567781 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:58:36.567781 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:58:36.567781 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:58:36.567781 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:58:36.567781 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:58:36.567781 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:58:36.567781 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:58:36.567781 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:58:36.567781 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:58:36.567781 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:58:36.567781 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:58:36.567781 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 18 04:58:36.567781 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 18 04:58:36.567781 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 18 04:58:36.567781 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 18 04:58:36.570096 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 18 04:58:36.570096 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 18 04:58:36.570096 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 18 04:58:36.570096 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 18 04:58:36.570096 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:58:36.570096 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:58:36.570096 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:58:36.570096 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:58:36.570096 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:58:36.570096 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:58:36.570096 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:58:36.570096 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jun 18 04:58:36.570096 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:58:36.570096 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:58:36.570096 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:58:36.570096 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:58:36.570923 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:58:36.570923 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:58:36.570923 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:58:36.570923 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:58:36.570923 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 18 04:58:36.570923 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 18 04:58:36.570923 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 18 04:58:36.570923 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 18 04:58:36.570923 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 18 04:58:36.570923 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 18 04:58:36.570923 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 18 04:58:36.570923 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 04:58:36.576101 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 198/383] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Jun 18 04:58:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 18 04:58:36.581896 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf3d1b4-442c-4a22-999c-2db6a5c31f28 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:36.628438 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:36.703488 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf3d1b4-442c-4a22-999c-2db6a5c31f28 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:36.741229 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bbe4f5-d6f0-4b90-8100-474b4b37e8a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:36.742045 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bbe4f5-d6f0-4b90-8100-474b4b37e8a6 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:36.742202 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bbe4f5-d6f0-4b90-8100-474b4b37e8a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:36.742442 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bbe4f5-d6f0-4b90-8100-474b4b37e8a6 None admin] PATH_INFO: `/v3/roles` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:36.743197 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bbe4f5-d6f0-4b90-8100-474b4b37e8a6 None admin] RBAC: Adding query filter params (name=member) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:58:36.743500 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bbe4f5-d6f0-4b90-8100-474b4b37e8a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:36.744870 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bbe4f5-d6f0-4b90-8100-474b4b37e8a6 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:36.753935 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 186/384] 10.222.0.23 () {56 vars in 1101 bytes} [Sat Jun 18 04:58:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:36.759463 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffc4774-08f1-4d61-b4ef-5ece546ec15d None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:36.778755 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f91dcd8-7aa4-4086-afed-e04b0dca18d4 None None] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:36.779176 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f91dcd8-7aa4-4086-afed-e04b0dca18d4 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:36.779337 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f91dcd8-7aa4-4086-afed-e04b0dca18d4 None None] PATH_INFO: `/` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:36.780683 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 187/385] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 04:58:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 18 04:58:36.789806 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc00ea8-1a64-4d48-ac45-b342bd326ccd None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:36.790100 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc00ea8-1a64-4d48-ac45-b342bd326ccd None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:36.790387 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc00ea8-1a64-4d48-ac45-b342bd326ccd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:36.813587 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-4cc00ea8-1a64-4d48-ac45-b342bd326ccd None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 04:58:36.814798 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-4cc00ea8-1a64-4d48-ac45-b342bd326ccd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 04:58:36.889405 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffc4774-08f1-4d61-b4ef-5ece546ec15d None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:36.931736 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e828417f-a577-4d25-a732-909bdeb3b9f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:36.932612 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e828417f-a577-4d25-a732-909bdeb3b9f0 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:36.932882 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e828417f-a577-4d25-a732-909bdeb3b9f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:36.933159 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e828417f-a577-4d25-a732-909bdeb3b9f0 None admin] PATH_INFO: `/v3/groups/0edcd696603f45549001165885fb7a69` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:36.936424 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e828417f-a577-4d25-a732-909bdeb3b9f0 None admin] RBAC: Authorizing `identity:get_group(group_id=0edcd696603f45549001165885fb7a69)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:36.937913 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e828417f-a577-4d25-a732-909bdeb3b9f0 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:36.946130 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 188/386] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 04:58:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 18 04:58:36.946389 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 199/387] 10.222.0.23 () {56 vars in 1180 bytes} [Sat Jun 18 04:58:36 2022] GET /identity/v3/groups/0edcd696603f45549001165885fb7a69 => generated 226 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:36.959799 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc00ea8-1a64-4d48-ac45-b342bd326ccd None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:37.003242 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:37.017207 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e828417f-a577-4d25-a732-909bdeb3b9f0 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:37.061965 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:37.098767 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc00ea8-1a64-4d48-ac45-b342bd326ccd None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:37.142182 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1689698b-7c1a-4efa-ae3c-b8c57c1e21d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:37.142980 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1689698b-7c1a-4efa-ae3c-b8c57c1e21d4 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:37.143265 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1689698b-7c1a-4efa-ae3c-b8c57c1e21d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:37.143543 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1689698b-7c1a-4efa-ae3c-b8c57c1e21d4 None admin] PATH_INFO: `/v3/projects/6706fdb556af472c8319dd2a13abe441` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:37.150212 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1689698b-7c1a-4efa-ae3c-b8c57c1e21d4 None admin] RBAC: Authorizing `identity:get_project(project_id=6706fdb556af472c8319dd2a13abe441)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:37.151668 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1689698b-7c1a-4efa-ae3c-b8c57c1e21d4 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:37.153804 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 189/388] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Jun 18 04:58:36 2022] GET /identity/v3/projects/6706fdb556af472c8319dd2a13abe441 => generated 298 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:37.161143 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1689698b-7c1a-4efa-ae3c-b8c57c1e21d4 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:37.165014 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e828417f-a577-4d25-a732-909bdeb3b9f0 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:37.230141 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee36f85-b22d-4a2d-aeef-58eb840c1e54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:37.230964 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee36f85-b22d-4a2d-aeef-58eb840c1e54 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:37.231246 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee36f85-b22d-4a2d-aeef-58eb840c1e54 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:37.231521 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee36f85-b22d-4a2d-aeef-58eb840c1e54 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:37.246686 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee36f85-b22d-4a2d-aeef-58eb840c1e54 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 18 04:58:37.263117 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-8ee36f85-b22d-4a2d-aeef-58eb840c1e54 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 18 04:58:37.263117 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:58:37.263117 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:58:37.263117 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:58:37.263117 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:58:37.263117 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:58:37.263117 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:58:37.263117 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:58:37.263117 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:58:37.263117 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:58:37.263117 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:58:37.263117 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:58:37.263117 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 18 04:58:37.263117 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 18 04:58:37.263117 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 18 04:58:37.263117 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 18 04:58:37.264049 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 18 04:58:37.264049 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 18 04:58:37.264049 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 18 04:58:37.264049 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 18 04:58:37.264049 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:58:37.264049 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:58:37.264049 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:58:37.264049 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:58:37.264049 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:58:37.264049 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:58:37.264049 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:58:37.264049 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jun 18 04:58:37.264049 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:58:37.264049 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:58:37.264049 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:58:37.264049 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:58:37.264831 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:58:37.264831 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:58:37.264831 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:58:37.264831 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:58:37.264831 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 18 04:58:37.264831 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 18 04:58:37.264831 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 18 04:58:37.264831 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 18 04:58:37.264831 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 18 04:58:37.264831 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 18 04:58:37.264831 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 18 04:58:37.264831 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 04:58:37.264831 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 200/389] 10.222.0.23 () {56 vars in 1114 bytes} [Sat Jun 18 04:58:37 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 18 04:58:37.268986 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:37.274400 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee36f85-b22d-4a2d-aeef-58eb840c1e54 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:37.278408 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1689698b-7c1a-4efa-ae3c-b8c57c1e21d4 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:37.318846 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc036f8b-e124-4a81-8417-fbf4f406b7f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:37.319914 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc036f8b-e124-4a81-8417-fbf4f406b7f8 None admin] REQUEST_METHOD: `PUT` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:37.320198 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc036f8b-e124-4a81-8417-fbf4f406b7f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:37.320474 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc036f8b-e124-4a81-8417-fbf4f406b7f8 None admin] PATH_INFO: `/v3/projects/6706fdb556af472c8319dd2a13abe441/groups/0edcd696603f45549001165885fb7a69/roles/cbf5c4c26898413d93e73b46b41046a3` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:37.324270 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc036f8b-e124-4a81-8417-fbf4f406b7f8 None admin] RBAC: Authorizing `identity:create_grant(project_id=6706fdb556af472c8319dd2a13abe441, group_id=0edcd696603f45549001165885fb7a69, role_id=cbf5c4c26898413d93e73b46b41046a3)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:37.325741 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc036f8b-e124-4a81-8417-fbf4f406b7f8 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:37.422035 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 190/390] 10.222.0.23 () {58 vars in 1442 bytes} [Sat Jun 18 04:58:37 2022] PUT /identity/v3/projects/6706fdb556af472c8319dd2a13abe441/groups/0edcd696603f45549001165885fb7a69/roles/cbf5c4c26898413d93e73b46b41046a3 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 18 04:58:37.424756 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:37.437212 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee36f85-b22d-4a2d-aeef-58eb840c1e54 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:37.482323 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a9bd6f-8239-40dd-82c1-cc17d9411d48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:37.483424 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a9bd6f-8239-40dd-82c1-cc17d9411d48 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:37.483955 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a9bd6f-8239-40dd-82c1-cc17d9411d48 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:37.484464 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a9bd6f-8239-40dd-82c1-cc17d9411d48 None admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:37.485439 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a9bd6f-8239-40dd-82c1-cc17d9411d48 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:58:37.485982 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a9bd6f-8239-40dd-82c1-cc17d9411d48 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:37.487562 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a9bd6f-8239-40dd-82c1-cc17d9411d48 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:37.500020 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 201/391] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Jun 18 04:58:37 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:37.505423 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc036f8b-e124-4a81-8417-fbf4f406b7f8 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:37.628879 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc036f8b-e124-4a81-8417-fbf4f406b7f8 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:37.670917 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd561c4a-e23d-486f-8451-e1710431100b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:37.672249 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd561c4a-e23d-486f-8451-e1710431100b None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:37.672758 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd561c4a-e23d-486f-8451-e1710431100b None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:37.673226 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd561c4a-e23d-486f-8451-e1710431100b None admin] PATH_INFO: `/v3/groups/0edcd696603f45549001165885fb7a69` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:37.674819 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd561c4a-e23d-486f-8451-e1710431100b None admin] RBAC: Authorizing `identity:get_group(group_id=0edcd696603f45549001165885fb7a69)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:37.676505 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd561c4a-e23d-486f-8451-e1710431100b None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:37.679989 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 191/392] 10.222.0.23 () {56 vars in 1180 bytes} [Sat Jun 18 04:58:37 2022] GET /identity/v3/groups/0edcd696603f45549001165885fb7a69 => generated 226 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:37.685205 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a9bd6f-8239-40dd-82c1-cc17d9411d48 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:37.824097 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a9bd6f-8239-40dd-82c1-cc17d9411d48 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:37.864970 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3093b467-d42a-4fc8-ad81-33a94837c66c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:37.867601 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3093b467-d42a-4fc8-ad81-33a94837c66c None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:37.868127 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3093b467-d42a-4fc8-ad81-33a94837c66c None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:37.868595 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3093b467-d42a-4fc8-ad81-33a94837c66c None admin] PATH_INFO: `/v3/projects/6706fdb556af472c8319dd2a13abe441` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:37.870033 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3093b467-d42a-4fc8-ad81-33a94837c66c None admin] RBAC: Authorizing `identity:get_project(project_id=6706fdb556af472c8319dd2a13abe441)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:37.871701 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3093b467-d42a-4fc8-ad81-33a94837c66c None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:37.874130 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 202/393] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Jun 18 04:58:37 2022] GET /identity/v3/projects/6706fdb556af472c8319dd2a13abe441 => generated 298 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:37.879448 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd561c4a-e23d-486f-8451-e1710431100b None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:37.921775 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:37.999728 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd561c4a-e23d-486f-8451-e1710431100b None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:38.036365 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a527f4-2266-4c45-958e-b4984a4cdd61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:38.038553 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a527f4-2266-4c45-958e-b4984a4cdd61 None admin] REQUEST_METHOD: `PUT` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:38.038940 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a527f4-2266-4c45-958e-b4984a4cdd61 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:38.039331 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a527f4-2266-4c45-958e-b4984a4cdd61 None admin] PATH_INFO: `/v3/projects/6706fdb556af472c8319dd2a13abe441/groups/0edcd696603f45549001165885fb7a69/roles/0089a3aad83c4ae08d8e9d864957285b` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:38.041710 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a527f4-2266-4c45-958e-b4984a4cdd61 None admin] RBAC: Authorizing `identity:create_grant(project_id=6706fdb556af472c8319dd2a13abe441, group_id=0edcd696603f45549001165885fb7a69, role_id=0089a3aad83c4ae08d8e9d864957285b)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:38.043239 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a527f4-2266-4c45-958e-b4984a4cdd61 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:38.144284 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 192/394] 10.222.0.23 () {58 vars in 1442 bytes} [Sat Jun 18 04:58:37 2022] PUT /identity/v3/projects/6706fdb556af472c8319dd2a13abe441/groups/0edcd696603f45549001165885fb7a69/roles/0089a3aad83c4ae08d8e9d864957285b => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 04:58:38.370472 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea52395-d966-4842-8310-0f6c7c1bc43f None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:38.370883 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea52395-d966-4842-8310-0f6c7c1bc43f None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:38.371264 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea52395-d966-4842-8310-0f6c7c1bc43f None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:38.377999 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 203/395] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 04:58:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 04:58:38.379668 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4b1bde-a85c-458b-8e28-7a19f45f93e7 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:38.379960 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4b1bde-a85c-458b-8e28-7a19f45f93e7 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:38.380241 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4b1bde-a85c-458b-8e28-7a19f45f93e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:38.399306 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-7d4b1bde-a85c-458b-8e28-7a19f45f93e7 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 04:58:38.400529 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-7d4b1bde-a85c-458b-8e28-7a19f45f93e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 04:58:38.483818 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 193/396] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 04:58:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 18 04:58:38.506325 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea52395-d966-4842-8310-0f6c7c1bc43f None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:38.541278 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:38.654159 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea52395-d966-4842-8310-0f6c7c1bc43f None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:38.697057 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e522d9d6-12a2-4458-97dd-db4b8b050d35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:38.697903 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e522d9d6-12a2-4458-97dd-db4b8b050d35 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:38.698190 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e522d9d6-12a2-4458-97dd-db4b8b050d35 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:38.698474 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e522d9d6-12a2-4458-97dd-db4b8b050d35 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:38.714886 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e522d9d6-12a2-4458-97dd-db4b8b050d35 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 18 04:58:38.722686 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-e522d9d6-12a2-4458-97dd-db4b8b050d35 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 18 04:58:38.722686 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:58:38.722686 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:58:38.722686 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:58:38.722686 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:58:38.722686 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:58:38.722686 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:58:38.722686 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:58:38.722686 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:58:38.722686 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:58:38.722686 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:58:38.722686 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:58:38.722686 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 18 04:58:38.722686 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 18 04:58:38.722686 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 18 04:58:38.722686 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 18 04:58:38.726985 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 18 04:58:38.726985 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 18 04:58:38.726985 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 18 04:58:38.726985 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 18 04:58:38.726985 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:58:38.726985 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:58:38.726985 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:58:38.726985 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:58:38.726985 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:58:38.726985 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:58:38.726985 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:58:38.726985 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jun 18 04:58:38.726985 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:58:38.726985 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:58:38.726985 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:58:38.726985 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:58:38.727764 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:58:38.727764 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:58:38.727764 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:58:38.727764 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:58:38.727764 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 18 04:58:38.727764 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 18 04:58:38.727764 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 18 04:58:38.727764 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 18 04:58:38.727764 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 18 04:58:38.727764 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 18 04:58:38.727764 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 18 04:58:38.727764 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 04:58:38.727764 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 204/397] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Jun 18 04:58:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:58:38.730718 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4b1bde-a85c-458b-8e28-7a19f45f93e7 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:38.762535 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:38.873627 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4b1bde-a85c-458b-8e28-7a19f45f93e7 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:38.916648 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b43accc-7525-484a-938d-c5d5995b6d64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:38.917292 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b43accc-7525-484a-938d-c5d5995b6d64 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:38.917601 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b43accc-7525-484a-938d-c5d5995b6d64 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:38.917874 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b43accc-7525-484a-938d-c5d5995b6d64 None admin] PATH_INFO: `/v3/roles` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:38.918685 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b43accc-7525-484a-938d-c5d5995b6d64 None admin] RBAC: Adding query filter params (name=member) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:58:38.918995 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b43accc-7525-484a-938d-c5d5995b6d64 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:38.920463 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b43accc-7525-484a-938d-c5d5995b6d64 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:38.933810 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 194/398] 10.222.0.23 () {56 vars in 1101 bytes} [Sat Jun 18 04:58:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:38.938927 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e522d9d6-12a2-4458-97dd-db4b8b050d35 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:39.002423 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e24ea17-94ca-4452-92fc-82e1cf017b0b None None] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:39.002756 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e24ea17-94ca-4452-92fc-82e1cf017b0b None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:39.002990 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e24ea17-94ca-4452-92fc-82e1cf017b0b None None] PATH_INFO: `/` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:39.005597 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 195/399] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 04:58:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 18 04:58:39.011052 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137dc538-4e9e-4e77-9325-b84b327f7090 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:39.011453 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137dc538-4e9e-4e77-9325-b84b327f7090 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:39.011739 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137dc538-4e9e-4e77-9325-b84b327f7090 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:39.029744 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-137dc538-4e9e-4e77-9325-b84b327f7090 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 04:58:39.031061 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-137dc538-4e9e-4e77-9325-b84b327f7090 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 04:58:39.054818 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:39.081067 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e522d9d6-12a2-4458-97dd-db4b8b050d35 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:39.113525 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 196/400] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 04:58:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 18 04:58:39.135787 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5439dd0-8e84-4236-ab46-988fad5dd008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:39.136005 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137dc538-4e9e-4e77-9325-b84b327f7090 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:39.136650 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5439dd0-8e84-4236-ab46-988fad5dd008 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:39.136933 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5439dd0-8e84-4236-ab46-988fad5dd008 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:39.137209 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5439dd0-8e84-4236-ab46-988fad5dd008 None admin] PATH_INFO: `/v3/groups/0edcd696603f45549001165885fb7a69` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:39.142103 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5439dd0-8e84-4236-ab46-988fad5dd008 None admin] RBAC: Authorizing `identity:get_group(group_id=0edcd696603f45549001165885fb7a69)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:39.143563 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5439dd0-8e84-4236-ab46-988fad5dd008 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:39.145535 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 205/401] 10.222.0.23 () {56 vars in 1180 bytes} [Sat Jun 18 04:58:38 2022] GET /identity/v3/groups/0edcd696603f45549001165885fb7a69 => generated 226 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:39.151401 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5439dd0-8e84-4236-ab46-988fad5dd008 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:39.151993 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:39.253873 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137dc538-4e9e-4e77-9325-b84b327f7090 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:39.295163 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffd19b0-ee7f-4716-acf6-953890910232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:39.295968 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffd19b0-ee7f-4716-acf6-953890910232 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:39.296253 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffd19b0-ee7f-4716-acf6-953890910232 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:39.296529 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffd19b0-ee7f-4716-acf6-953890910232 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:39.308949 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffd19b0-ee7f-4716-acf6-953890910232 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 18 04:58:39.313233 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5439dd0-8e84-4236-ab46-988fad5dd008 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:39.317019 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application [None req-fffd19b0-ee7f-4716-acf6-953890910232 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 18 04:58:39.317019 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:58:39.317019 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:58:39.317019 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:58:39.317019 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:58:39.317019 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:58:39.317019 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:58:39.317019 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:58:39.317019 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:58:39.317019 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:58:39.317019 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:58:39.317019 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:58:39.317019 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 18 04:58:39.317019 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 18 04:58:39.317019 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 18 04:58:39.317019 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 18 04:58:39.317912 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 18 04:58:39.317912 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 18 04:58:39.317912 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 18 04:58:39.317912 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 18 04:58:39.317912 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:58:39.317912 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:58:39.317912 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:58:39.317912 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:58:39.317912 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:58:39.317912 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:58:39.317912 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:58:39.317912 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application with Lock( Jun 18 04:58:39.317912 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:58:39.317912 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:58:39.317912 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:58:39.317912 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:58:39.318809 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:58:39.318809 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:58:39.318809 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:58:39.318809 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:58:39.318809 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 18 04:58:39.318809 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 18 04:58:39.318809 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 18 04:58:39.318809 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 18 04:58:39.318809 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 18 04:58:39.318809 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 18 04:58:39.318809 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 18 04:58:39.318809 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 04:58:39.323560 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 197/402] 10.222.0.23 () {56 vars in 1114 bytes} [Sat Jun 18 04:58:39 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:58:39.324882 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d63dd9-8726-495d-a007-8b0d815dfbd0 None None] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:39.325174 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d63dd9-8726-495d-a007-8b0d815dfbd0 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:39.325480 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d63dd9-8726-495d-a007-8b0d815dfbd0 None None] PATH_INFO: `/` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:39.326770 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 198/403] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 04:58:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 04:58:39.331033 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d63dd9-8726-495d-a007-8b0d815dfbd0 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:39.365172 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a76535-a941-434b-8c36-fde2c570a74e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:39.366959 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a76535-a941-434b-8c36-fde2c570a74e None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:39.367577 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a76535-a941-434b-8c36-fde2c570a74e None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:39.368136 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a76535-a941-434b-8c36-fde2c570a74e None admin] PATH_INFO: `/v3/projects/41cb6d0c536f4c4494fdabc03643aef0` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:39.369868 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a76535-a941-434b-8c36-fde2c570a74e None admin] RBAC: Authorizing `identity:get_project(project_id=41cb6d0c536f4c4494fdabc03643aef0)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:39.372395 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a76535-a941-434b-8c36-fde2c570a74e None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:39.374660 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 206/404] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Jun 18 04:58:39 2022] GET /identity/v3/projects/41cb6d0c536f4c4494fdabc03643aef0 => generated 302 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:39.376646 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0c9221-8ff8-461e-827e-92630fb9effc None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:39.377124 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0c9221-8ff8-461e-827e-92630fb9effc None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:39.378038 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0c9221-8ff8-461e-827e-92630fb9effc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:39.405402 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-db0c9221-8ff8-461e-827e-92630fb9effc None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 04:58:39.405402 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-db0c9221-8ff8-461e-827e-92630fb9effc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 04:58:39.405701 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:39.422599 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:39.446538 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d63dd9-8726-495d-a007-8b0d815dfbd0 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:39.487684 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb16fe6-61df-40d4-ba97-a8ad43278799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:39.488642 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb16fe6-61df-40d4-ba97-a8ad43278799 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:39.493454 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 207/405] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 04:58:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 18 04:58:39.493454 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0c9221-8ff8-461e-827e-92630fb9effc None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:39.493812 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb16fe6-61df-40d4-ba97-a8ad43278799 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:39.494057 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb16fe6-61df-40d4-ba97-a8ad43278799 None admin] PATH_INFO: `/v3/roles` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:39.494847 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb16fe6-61df-40d4-ba97-a8ad43278799 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:58:39.497199 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb16fe6-61df-40d4-ba97-a8ad43278799 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:39.500987 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb16fe6-61df-40d4-ba97-a8ad43278799 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:39.516743 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 199/406] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Jun 18 04:58:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:39.518671 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb16fe6-61df-40d4-ba97-a8ad43278799 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:39.610891 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0c9221-8ff8-461e-827e-92630fb9effc None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:39.640304 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb16fe6-61df-40d4-ba97-a8ad43278799 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:39.653332 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1551ba30-fd4b-4174-ba98-ab8404c26163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:39.654531 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1551ba30-fd4b-4174-ba98-ab8404c26163 None admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:39.655046 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1551ba30-fd4b-4174-ba98-ab8404c26163 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:39.655544 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1551ba30-fd4b-4174-ba98-ab8404c26163 None admin] PATH_INFO: `/v3/projects/41cb6d0c536f4c4494fdabc03643aef0/groups/0edcd696603f45549001165885fb7a69/roles/cbf5c4c26898413d93e73b46b41046a3` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:39.657783 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1551ba30-fd4b-4174-ba98-ab8404c26163 None admin] RBAC: Authorizing `identity:create_grant(project_id=41cb6d0c536f4c4494fdabc03643aef0, group_id=0edcd696603f45549001165885fb7a69, role_id=cbf5c4c26898413d93e73b46b41046a3)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:39.659427 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1551ba30-fd4b-4174-ba98-ab8404c26163 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:39.674566 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:39.682705 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6cb605-e47c-417a-bef3-22e550be4fbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:39.685052 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6cb605-e47c-417a-bef3-22e550be4fbb None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:39.685052 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6cb605-e47c-417a-bef3-22e550be4fbb None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:39.685052 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6cb605-e47c-417a-bef3-22e550be4fbb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:39.695510 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6cb605-e47c-417a-bef3-22e550be4fbb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 18 04:58:39.703233 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application [None req-4b6cb605-e47c-417a-bef3-22e550be4fbb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 18 04:58:39.703233 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:58:39.703233 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:58:39.703233 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:58:39.703233 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:58:39.703233 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:58:39.703233 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:58:39.703233 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:58:39.703233 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:58:39.703233 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:58:39.703233 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:58:39.703233 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:58:39.703233 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 18 04:58:39.703233 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 18 04:58:39.703233 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 18 04:58:39.703233 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 18 04:58:39.704127 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 18 04:58:39.704127 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 18 04:58:39.704127 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 18 04:58:39.704127 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 18 04:58:39.704127 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:58:39.704127 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:58:39.704127 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:58:39.704127 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:58:39.704127 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:58:39.704127 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:58:39.704127 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:58:39.704127 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application with Lock( Jun 18 04:58:39.704127 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:58:39.704127 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:58:39.704127 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:58:39.704127 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:58:39.704906 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:58:39.704906 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:58:39.704906 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:58:39.704906 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:58:39.704906 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 18 04:58:39.704906 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 18 04:58:39.704906 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 18 04:58:39.704906 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 18 04:58:39.704906 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 18 04:58:39.704906 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 18 04:58:39.704906 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 18 04:58:39.704906 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 04:58:39.704906 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 200/407] 10.222.0.23 () {56 vars in 1096 bytes} [Sat Jun 18 04:58:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:58:39.705623 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6cb605-e47c-417a-bef3-22e550be4fbb None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:39.767831 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 208/408] 10.222.0.23 () {58 vars in 1442 bytes} [Sat Jun 18 04:58:39 2022] PUT /identity/v3/projects/41cb6d0c536f4c4494fdabc03643aef0/groups/0edcd696603f45549001165885fb7a69/roles/cbf5c4c26898413d93e73b46b41046a3 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 04:58:39.768973 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1551ba30-fd4b-4174-ba98-ab8404c26163 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:39.789182 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:39.885692 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6cb605-e47c-417a-bef3-22e550be4fbb None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:39.950251 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1551ba30-fd4b-4174-ba98-ab8404c26163 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:39.980661 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f0c283-2e33-414f-a9e4-369fb912fe32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:39.981620 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f0c283-2e33-414f-a9e4-369fb912fe32 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:39.981909 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f0c283-2e33-414f-a9e4-369fb912fe32 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:39.982211 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f0c283-2e33-414f-a9e4-369fb912fe32 None admin] PATH_INFO: `/v3/groups/0edcd696603f45549001165885fb7a69` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:39.983447 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f0c283-2e33-414f-a9e4-369fb912fe32 None admin] RBAC: Authorizing `identity:get_group(group_id=0edcd696603f45549001165885fb7a69)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:39.984917 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f0c283-2e33-414f-a9e4-369fb912fe32 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:39.986857 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 201/409] 10.222.0.23 () {56 vars in 1180 bytes} [Sat Jun 18 04:58:39 2022] GET /identity/v3/groups/0edcd696603f45549001165885fb7a69 => generated 226 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:39.993088 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f0c283-2e33-414f-a9e4-369fb912fe32 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:39.994725 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0688a4-ac5b-4339-9e76-6c48c2115af5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:39.996007 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0688a4-ac5b-4339-9e76-6c48c2115af5 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:39.996496 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0688a4-ac5b-4339-9e76-6c48c2115af5 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:39.998092 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0688a4-ac5b-4339-9e76-6c48c2115af5 None admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:39.999106 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0688a4-ac5b-4339-9e76-6c48c2115af5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:58:39.999630 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0688a4-ac5b-4339-9e76-6c48c2115af5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:40.001240 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0688a4-ac5b-4339-9e76-6c48c2115af5 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:40.020208 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 209/410] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Jun 18 04:58:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:40.022332 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0688a4-ac5b-4339-9e76-6c48c2115af5 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:40.070159 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:40.106413 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:40.129661 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f0c283-2e33-414f-a9e4-369fb912fe32 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:40.160597 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0688a4-ac5b-4339-9e76-6c48c2115af5 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:40.172059 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79b1253-d944-422a-8728-646c8469b89f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:40.173217 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79b1253-d944-422a-8728-646c8469b89f None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:40.173719 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79b1253-d944-422a-8728-646c8469b89f None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:40.174222 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79b1253-d944-422a-8728-646c8469b89f None admin] PATH_INFO: `/v3/projects/41cb6d0c536f4c4494fdabc03643aef0` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:40.175614 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79b1253-d944-422a-8728-646c8469b89f None admin] RBAC: Authorizing `identity:get_project(project_id=41cb6d0c536f4c4494fdabc03643aef0)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:40.177263 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79b1253-d944-422a-8728-646c8469b89f None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:40.180043 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 202/411] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Jun 18 04:58:39 2022] GET /identity/v3/projects/41cb6d0c536f4c4494fdabc03643aef0 => generated 302 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 18 04:58:40.185847 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79b1253-d944-422a-8728-646c8469b89f None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:40.219468 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2143fe43-2e33-44a6-9bfe-d6a0559b13c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:40.221085 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2143fe43-2e33-44a6-9bfe-d6a0559b13c2 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:40.221369 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2143fe43-2e33-44a6-9bfe-d6a0559b13c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:40.221678 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2143fe43-2e33-44a6-9bfe-d6a0559b13c2 None admin] PATH_INFO: `/v3/groups/6e930b1dfac546d6a830d4cc1808c375` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:40.223422 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2143fe43-2e33-44a6-9bfe-d6a0559b13c2 None admin] RBAC: Authorizing `identity:get_group(group_id=6e930b1dfac546d6a830d4cc1808c375)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:40.226384 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2143fe43-2e33-44a6-9bfe-d6a0559b13c2 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:40.229495 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 210/412] 10.222.0.23 () {56 vars in 1180 bytes} [Sat Jun 18 04:58:40 2022] GET /identity/v3/groups/6e930b1dfac546d6a830d4cc1808c375 => generated 229 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:40.235189 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af156b3-5ec5-4bc4-baa2-b577a5290446 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:40.235651 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af156b3-5ec5-4bc4-baa2-b577a5290446 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:40.236159 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af156b3-5ec5-4bc4-baa2-b577a5290446 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:40.238539 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 211/413] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 04:58:40 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 04:58:40.243690 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af156b3-5ec5-4bc4-baa2-b577a5290446 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:40.325934 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79b1253-d944-422a-8728-646c8469b89f None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:40.359120 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:40.360056 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af156b3-5ec5-4bc4-baa2-b577a5290446 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:40.380201 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb1ff95-c4ae-4c24-aafa-c6ba1d72222f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:40.381954 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb1ff95-c4ae-4c24-aafa-c6ba1d72222f None admin] REQUEST_METHOD: `PUT` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:40.385145 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb1ff95-c4ae-4c24-aafa-c6ba1d72222f None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:40.385438 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb1ff95-c4ae-4c24-aafa-c6ba1d72222f None admin] PATH_INFO: `/v3/projects/41cb6d0c536f4c4494fdabc03643aef0/groups/0edcd696603f45549001165885fb7a69/roles/0089a3aad83c4ae08d8e9d864957285b` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:40.388518 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb1ff95-c4ae-4c24-aafa-c6ba1d72222f None admin] RBAC: Authorizing `identity:create_grant(project_id=41cb6d0c536f4c4494fdabc03643aef0, group_id=0edcd696603f45549001165885fb7a69, role_id=0089a3aad83c4ae08d8e9d864957285b)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:40.391622 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb1ff95-c4ae-4c24-aafa-c6ba1d72222f None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:40.405365 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9502d1ba-ed13-4eda-a2d1-7b67d6cf1aa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:40.406254 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9502d1ba-ed13-4eda-a2d1-7b67d6cf1aa0 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:40.406531 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9502d1ba-ed13-4eda-a2d1-7b67d6cf1aa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:40.407473 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9502d1ba-ed13-4eda-a2d1-7b67d6cf1aa0 None admin] PATH_INFO: `/v3/projects/d587d7347f6f4de2882df28dc6b6a574` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:40.408012 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9502d1ba-ed13-4eda-a2d1-7b67d6cf1aa0 None admin] RBAC: Authorizing `identity:get_project(project_id=d587d7347f6f4de2882df28dc6b6a574)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:40.409468 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9502d1ba-ed13-4eda-a2d1-7b67d6cf1aa0 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:40.411530 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 212/414] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Jun 18 04:58:40 2022] GET /identity/v3/projects/d587d7347f6f4de2882df28dc6b6a574 => generated 344 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:40.412987 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10430704-51dd-4d83-ba8f-15b8e6bb468a None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:40.413302 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10430704-51dd-4d83-ba8f-15b8e6bb468a None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:40.413559 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10430704-51dd-4d83-ba8f-15b8e6bb468a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:40.425675 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:40.431266 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-10430704-51dd-4d83-ba8f-15b8e6bb468a None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 04:58:40.432446 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-10430704-51dd-4d83-ba8f-15b8e6bb468a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 04:58:40.461939 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 203/415] 10.222.0.23 () {58 vars in 1442 bytes} [Sat Jun 18 04:58:40 2022] PUT /identity/v3/projects/41cb6d0c536f4c4494fdabc03643aef0/groups/0edcd696603f45549001165885fb7a69/roles/0089a3aad83c4ae08d8e9d864957285b => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 04:58:40.463786 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb1ff95-c4ae-4c24-aafa-c6ba1d72222f None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:40.532582 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 213/416] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 04:58:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 18 04:58:40.575261 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10430704-51dd-4d83-ba8f-15b8e6bb468a None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:40.590966 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb1ff95-c4ae-4c24-aafa-c6ba1d72222f None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:40.633498 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59dd256-8421-4aa8-b09f-65a6b80f9688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:40.634831 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59dd256-8421-4aa8-b09f-65a6b80f9688 None admin] REQUEST_METHOD: `PUT` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:40.635322 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59dd256-8421-4aa8-b09f-65a6b80f9688 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:40.635864 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59dd256-8421-4aa8-b09f-65a6b80f9688 None admin] PATH_INFO: `/v3/projects/d587d7347f6f4de2882df28dc6b6a574/groups/6e930b1dfac546d6a830d4cc1808c375/roles/21c65f2c51bd4dd283d13507d2714467` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:40.638090 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59dd256-8421-4aa8-b09f-65a6b80f9688 None admin] RBAC: Authorizing `identity:create_grant(project_id=d587d7347f6f4de2882df28dc6b6a574, group_id=6e930b1dfac546d6a830d4cc1808c375, role_id=21c65f2c51bd4dd283d13507d2714467)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:40.639756 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59dd256-8421-4aa8-b09f-65a6b80f9688 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:40.710923 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10430704-51dd-4d83-ba8f-15b8e6bb468a None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:40.750935 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 204/417] 10.222.0.23 () {58 vars in 1442 bytes} [Sat Jun 18 04:58:40 2022] PUT /identity/v3/projects/d587d7347f6f4de2882df28dc6b6a574/groups/6e930b1dfac546d6a830d4cc1808c375/roles/21c65f2c51bd4dd283d13507d2714467 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 04:58:40.757494 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7f13c6-c474-4cef-8a03-def3e7508be8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:40.758736 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7f13c6-c474-4cef-8a03-def3e7508be8 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:40.759256 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7f13c6-c474-4cef-8a03-def3e7508be8 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:40.759749 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7f13c6-c474-4cef-8a03-def3e7508be8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:40.764185 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:40.774036 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7f13c6-c474-4cef-8a03-def3e7508be8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 18 04:58:40.782573 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-5a7f13c6-c474-4cef-8a03-def3e7508be8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 18 04:58:40.782573 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:58:40.782573 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:58:40.782573 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:58:40.782573 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:58:40.782573 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:58:40.782573 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:58:40.782573 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:58:40.782573 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:58:40.782573 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:58:40.782573 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:58:40.782573 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:58:40.782573 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 18 04:58:40.782573 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 18 04:58:40.782573 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 18 04:58:40.782573 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 18 04:58:40.786541 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 18 04:58:40.786541 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 18 04:58:40.786541 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 18 04:58:40.786541 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 18 04:58:40.786541 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:58:40.786541 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:58:40.786541 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:58:40.786541 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:58:40.786541 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:58:40.786541 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:58:40.786541 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:58:40.786541 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jun 18 04:58:40.786541 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:58:40.786541 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:58:40.786541 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:58:40.786541 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:58:40.787931 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:58:40.787931 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:58:40.787931 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:58:40.787931 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:58:40.787931 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 18 04:58:40.787931 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 18 04:58:40.787931 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 18 04:58:40.787931 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 18 04:58:40.787931 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 18 04:58:40.787931 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 18 04:58:40.787931 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 18 04:58:40.787931 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 04:58:40.787931 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 214/418] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Jun 18 04:58:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 18 04:58:40.790669 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59dd256-8421-4aa8-b09f-65a6b80f9688 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:40.863005 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:40.870929 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b09d8a7-57f4-4a04-b968-87c3486ab9e3 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:40.871223 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b09d8a7-57f4-4a04-b968-87c3486ab9e3 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:40.871503 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b09d8a7-57f4-4a04-b968-87c3486ab9e3 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:40.873072 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 215/419] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 04:58:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 04:58:40.882036 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf1ebe8-b496-4070-b8f7-418870f87a8d None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:40.882322 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf1ebe8-b496-4070-b8f7-418870f87a8d None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:40.882601 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf1ebe8-b496-4070-b8f7-418870f87a8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:40.916495 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-5bf1ebe8-b496-4070-b8f7-418870f87a8d None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 04:58:40.920670 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-5bf1ebe8-b496-4070-b8f7-418870f87a8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 04:58:40.951189 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59dd256-8421-4aa8-b09f-65a6b80f9688 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:40.987725 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fdae48-b448-4600-b7eb-593879e9437d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:40.988396 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fdae48-b448-4600-b7eb-593879e9437d None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:40.988681 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fdae48-b448-4600-b7eb-593879e9437d None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:40.988956 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fdae48-b448-4600-b7eb-593879e9437d None admin] PATH_INFO: `/v3/roles` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:40.989992 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fdae48-b448-4600-b7eb-593879e9437d None admin] RBAC: Adding query filter params (name=member) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:58:40.990373 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fdae48-b448-4600-b7eb-593879e9437d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:40.991820 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fdae48-b448-4600-b7eb-593879e9437d None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:41.000744 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 205/420] 10.222.0.23 () {56 vars in 1101 bytes} [Sat Jun 18 04:58:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:41.007274 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fdae48-b448-4600-b7eb-593879e9437d None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:41.029471 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 216/421] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 04:58:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 18 04:58:41.048600 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf1ebe8-b496-4070-b8f7-418870f87a8d None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:41.145408 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fdae48-b448-4600-b7eb-593879e9437d None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:41.162743 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:41.202892 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821a9207-b97f-4e4e-9043-429a9ce7bed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:41.204337 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821a9207-b97f-4e4e-9043-429a9ce7bed2 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:41.204846 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821a9207-b97f-4e4e-9043-429a9ce7bed2 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:41.205327 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821a9207-b97f-4e4e-9043-429a9ce7bed2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:41.252009 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf1ebe8-b496-4070-b8f7-418870f87a8d None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:41.307416 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:41.317837 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725bba90-f50f-425f-a9ab-d7082a79b7ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:41.318679 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821a9207-b97f-4e4e-9043-429a9ce7bed2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:41.318797 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725bba90-f50f-425f-a9ab-d7082a79b7ab None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:41.319074 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725bba90-f50f-425f-a9ab-d7082a79b7ab None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:41.319358 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725bba90-f50f-425f-a9ab-d7082a79b7ab None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:41.321439 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821a9207-b97f-4e4e-9043-429a9ce7bed2 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:41.331171 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725bba90-f50f-425f-a9ab-d7082a79b7ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 18 04:58:41.340911 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-725bba90-f50f-425f-a9ab-d7082a79b7ab None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 18 04:58:41.340911 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:58:41.340911 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:58:41.340911 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:58:41.340911 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:58:41.340911 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:58:41.340911 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:58:41.340911 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:58:41.340911 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:58:41.340911 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:58:41.340911 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:58:41.340911 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:58:41.340911 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 18 04:58:41.340911 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 18 04:58:41.340911 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 18 04:58:41.340911 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 18 04:58:41.341870 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 18 04:58:41.341870 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 18 04:58:41.341870 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 18 04:58:41.341870 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 18 04:58:41.341870 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:58:41.341870 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:58:41.341870 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:58:41.341870 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:58:41.341870 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:58:41.341870 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:58:41.341870 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:58:41.341870 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jun 18 04:58:41.341870 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:58:41.341870 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:58:41.341870 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:58:41.341870 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:58:41.343412 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:58:41.343412 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:58:41.343412 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:58:41.343412 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:58:41.343412 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 18 04:58:41.343412 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 18 04:58:41.343412 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 18 04:58:41.343412 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 18 04:58:41.343412 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 18 04:58:41.343412 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 18 04:58:41.343412 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 18 04:58:41.343412 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 04:58:41.343412 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 217/422] 10.222.0.23 () {56 vars in 1114 bytes} [Sat Jun 18 04:58:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 18 04:58:41.347720 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725bba90-f50f-425f-a9ab-d7082a79b7ab None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:41.496073 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 206/423] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Jun 18 04:58:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 488 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 18 04:58:41.503270 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821a9207-b97f-4e4e-9043-429a9ce7bed2 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:41.517759 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:41.567228 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725bba90-f50f-425f-a9ab-d7082a79b7ab None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:41.608106 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821a9207-b97f-4e4e-9043-429a9ce7bed2 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:41.643543 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0f6c73-f776-4ec4-9d63-4767cbdf38e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:41.644411 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0f6c73-f776-4ec4-9d63-4767cbdf38e9 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:41.644699 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0f6c73-f776-4ec4-9d63-4767cbdf38e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:41.644980 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0f6c73-f776-4ec4-9d63-4767cbdf38e9 None admin] PATH_INFO: `/v3/projects/6706fdb556af472c8319dd2a13abe441` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:41.646216 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0f6c73-f776-4ec4-9d63-4767cbdf38e9 None admin] RBAC: Authorizing `identity:get_project(project_id=6706fdb556af472c8319dd2a13abe441)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:41.647924 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0f6c73-f776-4ec4-9d63-4767cbdf38e9 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:41.650030 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 207/424] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Jun 18 04:58:41 2022] GET /identity/v3/projects/6706fdb556af472c8319dd2a13abe441 => generated 298 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:41.655544 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0f6c73-f776-4ec4-9d63-4767cbdf38e9 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:41.660399 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea04c35-61f5-480f-9679-30260d86d080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:41.661070 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea04c35-61f5-480f-9679-30260d86d080 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:41.661352 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea04c35-61f5-480f-9679-30260d86d080 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:41.661674 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea04c35-61f5-480f-9679-30260d86d080 None admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:41.662952 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea04c35-61f5-480f-9679-30260d86d080 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:58:41.663260 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea04c35-61f5-480f-9679-30260d86d080 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:41.664641 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea04c35-61f5-480f-9679-30260d86d080 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:41.682840 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 218/425] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Jun 18 04:58:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 18 04:58:41.687024 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea04c35-61f5-480f-9679-30260d86d080 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:41.789842 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:41.797740 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:41.850264 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0f6c73-f776-4ec4-9d63-4767cbdf38e9 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:41.877038 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea04c35-61f5-480f-9679-30260d86d080 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:41.922218 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6f7fb9-c7d6-4c27-9777-8a6469fe8499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:41.922859 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6f7fb9-c7d6-4c27-9777-8a6469fe8499 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:41.923136 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6f7fb9-c7d6-4c27-9777-8a6469fe8499 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:41.923408 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6f7fb9-c7d6-4c27-9777-8a6469fe8499 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:41.925170 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c95bcf-7af9-4f66-9899-4f5254c28772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:41.926052 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c95bcf-7af9-4f66-9899-4f5254c28772 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:41.926336 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c95bcf-7af9-4f66-9899-4f5254c28772 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:41.926617 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c95bcf-7af9-4f66-9899-4f5254c28772 None admin] PATH_INFO: `/v3/groups/0edcd696603f45549001165885fb7a69` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:41.928030 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c95bcf-7af9-4f66-9899-4f5254c28772 None admin] RBAC: Authorizing `identity:get_group(group_id=0edcd696603f45549001165885fb7a69)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:41.929550 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c95bcf-7af9-4f66-9899-4f5254c28772 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:41.933640 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 208/426] 10.222.0.23 () {56 vars in 1180 bytes} [Sat Jun 18 04:58:41 2022] GET /identity/v3/groups/0edcd696603f45549001165885fb7a69 => generated 226 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 18 04:58:41.938174 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c95bcf-7af9-4f66-9899-4f5254c28772 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:41.997788 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6f7fb9-c7d6-4c27-9777-8a6469fe8499 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:41.999246 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6f7fb9-c7d6-4c27-9777-8a6469fe8499 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:42.059330 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:42.101598 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 219/427] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Jun 18 04:58:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 18 04:58:42.107112 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6f7fb9-c7d6-4c27-9777-8a6469fe8499 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:42.197438 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c95bcf-7af9-4f66-9899-4f5254c28772 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:42.261137 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5245d479-3ee8-4d81-abbf-3df573bf4cea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:42.261813 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5245d479-3ee8-4d81-abbf-3df573bf4cea None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:42.262093 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5245d479-3ee8-4d81-abbf-3df573bf4cea None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:42.262365 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5245d479-3ee8-4d81-abbf-3df573bf4cea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:42.263193 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5245d479-3ee8-4d81-abbf-3df573bf4cea None admin] RBAC: Adding query filter params (group.id=0edcd696603f45549001165885fb7a69, role.id=cbf5c4c26898413d93e73b46b41046a3, scope.project.id=6706fdb556af472c8319dd2a13abe441) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:58:42.263503 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5245d479-3ee8-4d81-abbf-3df573bf4cea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:42.264925 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5245d479-3ee8-4d81-abbf-3df573bf4cea None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:42.273469 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 209/428] 10.222.0.23 () {56 vars in 1376 bytes} [Sat Jun 18 04:58:41 2022] GET /identity/v3/role_assignments?group.id=0edcd696603f45549001165885fb7a69&scope.project.id=6706fdb556af472c8319dd2a13abe441&role.id=cbf5c4c26898413d93e73b46b41046a3 => generated 611 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:42.308768 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6f7fb9-c7d6-4c27-9777-8a6469fe8499 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:42.363582 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3216ce-1a7e-41f0-a3fc-1f1adebb67e7 None None] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:42.364279 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3216ce-1a7e-41f0-a3fc-1f1adebb67e7 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:42.364606 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41549e45-54c1-40fc-a6e9-4b25722f334a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:42.365294 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3216ce-1a7e-41f0-a3fc-1f1adebb67e7 None None] PATH_INFO: `/` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:42.366036 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41549e45-54c1-40fc-a6e9-4b25722f334a None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:42.366408 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41549e45-54c1-40fc-a6e9-4b25722f334a None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:42.366687 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41549e45-54c1-40fc-a6e9-4b25722f334a None admin] PATH_INFO: `/v3/projects/6706fdb556af472c8319dd2a13abe441` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:42.368003 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 210/429] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 04:58:42 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 18 04:58:42.373766 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61512d85-6034-484c-a282-e7c5011be935 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:42.374310 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41549e45-54c1-40fc-a6e9-4b25722f334a None admin] RBAC: Authorizing `identity:get_project(project_id=6706fdb556af472c8319dd2a13abe441)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:42.374575 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61512d85-6034-484c-a282-e7c5011be935 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:42.375087 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61512d85-6034-484c-a282-e7c5011be935 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:42.375815 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41549e45-54c1-40fc-a6e9-4b25722f334a None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:42.377789 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 220/430] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Jun 18 04:58:42 2022] GET /identity/v3/projects/6706fdb556af472c8319dd2a13abe441 => generated 298 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:42.385963 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41549e45-54c1-40fc-a6e9-4b25722f334a None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:42.393930 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:42.400464 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-61512d85-6034-484c-a282-e7c5011be935 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 04:58:42.402027 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-61512d85-6034-484c-a282-e7c5011be935 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 04:58:42.418966 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:42.516143 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41549e45-54c1-40fc-a6e9-4b25722f334a None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:42.544373 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 211/431] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 04:58:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 18 04:58:42.563011 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415e3e77-4c4f-49b2-9c76-f70c5f5a98c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:42.563870 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415e3e77-4c4f-49b2-9c76-f70c5f5a98c3 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:42.564148 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415e3e77-4c4f-49b2-9c76-f70c5f5a98c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:42.564425 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415e3e77-4c4f-49b2-9c76-f70c5f5a98c3 None admin] PATH_INFO: `/v3/groups/0edcd696603f45549001165885fb7a69` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:42.565680 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415e3e77-4c4f-49b2-9c76-f70c5f5a98c3 None admin] RBAC: Authorizing `identity:get_group(group_id=0edcd696603f45549001165885fb7a69)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:42.567144 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415e3e77-4c4f-49b2-9c76-f70c5f5a98c3 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:42.569092 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 221/432] 10.222.0.23 () {56 vars in 1180 bytes} [Sat Jun 18 04:58:42 2022] GET /identity/v3/groups/0edcd696603f45549001165885fb7a69 => generated 226 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 18 04:58:42.570153 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61512d85-6034-484c-a282-e7c5011be935 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:42.575430 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415e3e77-4c4f-49b2-9c76-f70c5f5a98c3 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:42.670742 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:42.687610 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415e3e77-4c4f-49b2-9c76-f70c5f5a98c3 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:42.693982 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61512d85-6034-484c-a282-e7c5011be935 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:42.726828 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58865caf-f681-4467-af81-99b177155107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:42.727494 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58865caf-f681-4467-af81-99b177155107 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:42.727865 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58865caf-f681-4467-af81-99b177155107 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:42.728148 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58865caf-f681-4467-af81-99b177155107 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:42.728989 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58865caf-f681-4467-af81-99b177155107 None admin] RBAC: Adding query filter params (group.id=0edcd696603f45549001165885fb7a69, role.id=0089a3aad83c4ae08d8e9d864957285b, scope.project.id=6706fdb556af472c8319dd2a13abe441) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:58:42.729308 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58865caf-f681-4467-af81-99b177155107 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:42.730795 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58865caf-f681-4467-af81-99b177155107 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:42.738299 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e01242-913b-40a0-b2ec-332f46c28426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:42.741451 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 222/433] 10.222.0.23 () {56 vars in 1376 bytes} [Sat Jun 18 04:58:42 2022] GET /identity/v3/role_assignments?group.id=0edcd696603f45549001165885fb7a69&scope.project.id=6706fdb556af472c8319dd2a13abe441&role.id=0089a3aad83c4ae08d8e9d864957285b => generated 611 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:42.741560 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e01242-913b-40a0-b2ec-332f46c28426 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:42.741560 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e01242-913b-40a0-b2ec-332f46c28426 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:42.741560 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e01242-913b-40a0-b2ec-332f46c28426 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:42.747537 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:42.756208 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e01242-913b-40a0-b2ec-332f46c28426 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 18 04:58:42.764151 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application [None req-f7e01242-913b-40a0-b2ec-332f46c28426 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 18 04:58:42.764151 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:58:42.764151 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:58:42.764151 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:58:42.764151 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:58:42.764151 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:58:42.764151 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:58:42.764151 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:58:42.764151 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:58:42.764151 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:58:42.764151 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:58:42.764151 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:58:42.764151 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 18 04:58:42.764151 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 18 04:58:42.764151 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 18 04:58:42.764151 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 18 04:58:42.767140 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 18 04:58:42.767140 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 18 04:58:42.767140 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 18 04:58:42.767140 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 18 04:58:42.767140 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:58:42.767140 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:58:42.767140 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:58:42.767140 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:58:42.767140 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:58:42.767140 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:58:42.767140 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:58:42.767140 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application with Lock( Jun 18 04:58:42.767140 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:58:42.767140 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:58:42.767140 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:58:42.767140 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:58:42.767929 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:58:42.767929 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:58:42.767929 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:58:42.767929 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:58:42.767929 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 18 04:58:42.767929 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 18 04:58:42.767929 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 18 04:58:42.767929 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 18 04:58:42.767929 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 18 04:58:42.767929 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 18 04:58:42.767929 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 18 04:58:42.767929 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 04:58:42.767929 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 212/434] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Jun 18 04:58:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:58:42.771197 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58865caf-f681-4467-af81-99b177155107 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:42.876927 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89c07ca-9608-41e7-a57a-477b78ec8b2d None None] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:42.877632 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89c07ca-9608-41e7-a57a-477b78ec8b2d None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:42.878106 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89c07ca-9608-41e7-a57a-477b78ec8b2d None None] PATH_INFO: `/` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:42.879963 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 213/435] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 04:58:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 04:58:42.886618 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3436f423-8f1c-4f82-9304-66f7bcb68827 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:42.887294 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3436f423-8f1c-4f82-9304-66f7bcb68827 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:42.887763 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3436f423-8f1c-4f82-9304-66f7bcb68827 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:42.894625 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58865caf-f681-4467-af81-99b177155107 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:42.914280 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-3436f423-8f1c-4f82-9304-66f7bcb68827 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 04:58:42.915871 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-3436f423-8f1c-4f82-9304-66f7bcb68827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 04:58:42.938199 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4171ff79-bcab-4a50-8e63-6b2aefd32379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:42.938866 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4171ff79-bcab-4a50-8e63-6b2aefd32379 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:42.939152 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4171ff79-bcab-4a50-8e63-6b2aefd32379 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:42.939426 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4171ff79-bcab-4a50-8e63-6b2aefd32379 None admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:42.940264 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4171ff79-bcab-4a50-8e63-6b2aefd32379 None admin] RBAC: Adding query filter params (name=member) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:58:42.940577 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4171ff79-bcab-4a50-8e63-6b2aefd32379 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:42.942127 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4171ff79-bcab-4a50-8e63-6b2aefd32379 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:42.953424 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 223/436] 10.222.0.23 () {56 vars in 1101 bytes} [Sat Jun 18 04:58:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:42.961507 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4171ff79-bcab-4a50-8e63-6b2aefd32379 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:42.979480 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:43.012925 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 214/437] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 04:58:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 18 04:58:43.041814 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3436f423-8f1c-4f82-9304-66f7bcb68827 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:43.093039 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4171ff79-bcab-4a50-8e63-6b2aefd32379 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:43.142171 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f910ec4d-2653-41ae-96e4-49e2bdb034f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:43.142833 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f910ec4d-2653-41ae-96e4-49e2bdb034f9 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:43.143117 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f910ec4d-2653-41ae-96e4-49e2bdb034f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:43.143392 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f910ec4d-2653-41ae-96e4-49e2bdb034f9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:43.180194 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3436f423-8f1c-4f82-9304-66f7bcb68827 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:43.208758 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:43.223687 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5448dcc-f411-4deb-9b98-dc4ce49c38c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:43.224964 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5448dcc-f411-4deb-9b98-dc4ce49c38c3 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:43.225552 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5448dcc-f411-4deb-9b98-dc4ce49c38c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:43.225850 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5448dcc-f411-4deb-9b98-dc4ce49c38c3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:43.229078 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f910ec4d-2653-41ae-96e4-49e2bdb034f9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:43.230827 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f910ec4d-2653-41ae-96e4-49e2bdb034f9 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:43.242188 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5448dcc-f411-4deb-9b98-dc4ce49c38c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 18 04:58:43.245692 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:43.250781 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application [None req-a5448dcc-f411-4deb-9b98-dc4ce49c38c3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 18 04:58:43.250781 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:58:43.250781 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:58:43.250781 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:58:43.250781 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:58:43.250781 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:58:43.250781 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:58:43.250781 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:58:43.250781 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:58:43.250781 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:58:43.250781 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:58:43.250781 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:58:43.250781 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 18 04:58:43.250781 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 18 04:58:43.250781 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 18 04:58:43.250781 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 18 04:58:43.251827 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 18 04:58:43.251827 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 18 04:58:43.251827 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 18 04:58:43.251827 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 18 04:58:43.251827 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:58:43.251827 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:58:43.251827 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:58:43.251827 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:58:43.251827 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:58:43.251827 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:58:43.251827 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:58:43.251827 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application with Lock( Jun 18 04:58:43.251827 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:58:43.251827 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:58:43.251827 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:58:43.251827 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:58:43.252627 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:58:43.252627 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:58:43.252627 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:58:43.252627 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:58:43.252627 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 18 04:58:43.252627 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 18 04:58:43.252627 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 18 04:58:43.252627 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 18 04:58:43.252627 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 18 04:58:43.252627 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 18 04:58:43.252627 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 18 04:58:43.252627 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 04:58:43.255327 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 215/438] 10.222.0.23 () {56 vars in 1114 bytes} [Sat Jun 18 04:58:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:58:43.262800 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5448dcc-f411-4deb-9b98-dc4ce49c38c3 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:43.351739 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 224/439] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Jun 18 04:58:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 18 04:58:43.353113 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46b3715-f6c8-40fe-91d6-b0aa8dbe7dd5 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:43.357631 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46b3715-f6c8-40fe-91d6-b0aa8dbe7dd5 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:43.357943 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46b3715-f6c8-40fe-91d6-b0aa8dbe7dd5 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:43.359604 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 225/440] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 04:58:43 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 04:58:43.360383 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46b3715-f6c8-40fe-91d6-b0aa8dbe7dd5 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:43.385700 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5448dcc-f411-4deb-9b98-dc4ce49c38c3 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:43.421182 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdaa459-0821-4e52-85cd-c32b8071c40a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:43.421878 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdaa459-0821-4e52-85cd-c32b8071c40a None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:43.422171 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdaa459-0821-4e52-85cd-c32b8071c40a None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:43.422867 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdaa459-0821-4e52-85cd-c32b8071c40a None admin] PATH_INFO: `/v3/roles` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:43.423622 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdaa459-0821-4e52-85cd-c32b8071c40a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:58:43.423940 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdaa459-0821-4e52-85cd-c32b8071c40a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:43.425310 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdaa459-0821-4e52-85cd-c32b8071c40a None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:43.433905 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 216/441] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Jun 18 04:58:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:43.439215 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af71c601-b26f-4ef5-9839-5bd2a110c525 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:43.439508 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af71c601-b26f-4ef5-9839-5bd2a110c525 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:43.439790 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af71c601-b26f-4ef5-9839-5bd2a110c525 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:43.456238 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-af71c601-b26f-4ef5-9839-5bd2a110c525 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 04:58:43.457451 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-af71c601-b26f-4ef5-9839-5bd2a110c525 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 04:58:43.469352 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46b3715-f6c8-40fe-91d6-b0aa8dbe7dd5 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:43.479806 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:43.502104 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:43.503093 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0ae208-450e-4019-9743-c362bfab03ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:43.503945 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0ae208-450e-4019-9743-c362bfab03ea None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:43.504225 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0ae208-450e-4019-9743-c362bfab03ea None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:43.504497 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0ae208-450e-4019-9743-c362bfab03ea None admin] PATH_INFO: `/v3/projects/41cb6d0c536f4c4494fdabc03643aef0` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:43.505726 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0ae208-450e-4019-9743-c362bfab03ea None admin] RBAC: Authorizing `identity:get_project(project_id=41cb6d0c536f4c4494fdabc03643aef0)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:43.508288 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0ae208-450e-4019-9743-c362bfab03ea None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:43.509999 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 226/442] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Jun 18 04:58:43 2022] GET /identity/v3/projects/41cb6d0c536f4c4494fdabc03643aef0 => generated 302 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:43.510757 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0ae208-450e-4019-9743-c362bfab03ea None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:43.526348 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 217/443] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 04:58:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 18 04:58:43.526980 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af71c601-b26f-4ef5-9839-5bd2a110c525 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:43.626359 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0ae208-450e-4019-9743-c362bfab03ea None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:43.629410 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af71c601-b26f-4ef5-9839-5bd2a110c525 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:43.660044 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea479b5-d4a9-4e6a-ab3f-9d81bae5bdeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:43.660684 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea479b5-d4a9-4e6a-ab3f-9d81bae5bdeb None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:43.660967 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea479b5-d4a9-4e6a-ab3f-9d81bae5bdeb None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:43.661243 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea479b5-d4a9-4e6a-ab3f-9d81bae5bdeb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:43.664284 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37adbee-0935-412f-ac20-899e6b9dd23f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:43.665125 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37adbee-0935-412f-ac20-899e6b9dd23f None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:43.665433 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37adbee-0935-412f-ac20-899e6b9dd23f None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:43.665714 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37adbee-0935-412f-ac20-899e6b9dd23f None admin] PATH_INFO: `/v3/groups/0edcd696603f45549001165885fb7a69` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:43.666878 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37adbee-0935-412f-ac20-899e6b9dd23f None admin] RBAC: Authorizing `identity:get_group(group_id=0edcd696603f45549001165885fb7a69)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:43.668302 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37adbee-0935-412f-ac20-899e6b9dd23f None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:43.669947 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 218/444] 10.222.0.23 () {56 vars in 1180 bytes} [Sat Jun 18 04:58:43 2022] GET /identity/v3/groups/0edcd696603f45549001165885fb7a69 => generated 226 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:43.670695 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37adbee-0935-412f-ac20-899e6b9dd23f None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:43.707667 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:43.719208 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:43.729962 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea479b5-d4a9-4e6a-ab3f-9d81bae5bdeb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:43.731572 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea479b5-d4a9-4e6a-ab3f-9d81bae5bdeb None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:43.771506 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37adbee-0935-412f-ac20-899e6b9dd23f None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:43.805623 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd438e1-09d0-46be-be9d-c1458b057b51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:43.806403 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd438e1-09d0-46be-be9d-c1458b057b51 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:43.806689 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd438e1-09d0-46be-be9d-c1458b057b51 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:43.806966 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd438e1-09d0-46be-be9d-c1458b057b51 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:43.816911 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd438e1-09d0-46be-be9d-c1458b057b51 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 18 04:58:43.824520 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application [None req-bfd438e1-09d0-46be-be9d-c1458b057b51 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 18 04:58:43.824520 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:58:43.824520 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:58:43.824520 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:58:43.824520 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:58:43.824520 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:58:43.824520 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:58:43.824520 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:58:43.824520 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:58:43.824520 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:58:43.824520 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:58:43.824520 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:58:43.824520 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 18 04:58:43.824520 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 18 04:58:43.824520 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 18 04:58:43.824520 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 18 04:58:43.825420 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 18 04:58:43.825420 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 18 04:58:43.825420 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 18 04:58:43.825420 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 18 04:58:43.825420 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:58:43.825420 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:58:43.825420 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:58:43.825420 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:58:43.825420 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:58:43.825420 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:58:43.825420 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:58:43.825420 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application with Lock( Jun 18 04:58:43.825420 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:58:43.825420 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:58:43.825420 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:58:43.825420 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:58:43.826261 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:58:43.826261 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:58:43.826261 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:58:43.826261 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:58:43.826261 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 18 04:58:43.826261 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 18 04:58:43.826261 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 18 04:58:43.826261 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 18 04:58:43.826261 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 18 04:58:43.826261 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 18 04:58:43.826261 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 18 04:58:43.826261 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 04:58:43.826261 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 219/445] 10.222.0.23 () {56 vars in 1096 bytes} [Sat Jun 18 04:58:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:58:43.826913 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd438e1-09d0-46be-be9d-c1458b057b51 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:43.836094 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 227/446] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Jun 18 04:58:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 18 04:58:43.836914 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea479b5-d4a9-4e6a-ab3f-9d81bae5bdeb None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:43.913903 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:43.933291 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd438e1-09d0-46be-be9d-c1458b057b51 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:43.935525 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea479b5-d4a9-4e6a-ab3f-9d81bae5bdeb None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:43.940189 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:43.967704 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c84031-cbed-4697-9b86-074b7e76d6d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:43.968349 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c84031-cbed-4697-9b86-074b7e76d6d8 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:43.968639 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c84031-cbed-4697-9b86-074b7e76d6d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:43.968921 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c84031-cbed-4697-9b86-074b7e76d6d8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:43.969789 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c84031-cbed-4697-9b86-074b7e76d6d8 None admin] RBAC: Adding query filter params (group.id=0edcd696603f45549001165885fb7a69, role.id=cbf5c4c26898413d93e73b46b41046a3, scope.project.id=41cb6d0c536f4c4494fdabc03643aef0) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:58:43.970110 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c84031-cbed-4697-9b86-074b7e76d6d8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:43.970535 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dfaa4e-6bea-4ef9-9d18-bbcef6e4d444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:43.971194 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dfaa4e-6bea-4ef9-9d18-bbcef6e4d444 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:43.971477 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dfaa4e-6bea-4ef9-9d18-bbcef6e4d444 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:43.971564 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c84031-cbed-4697-9b86-074b7e76d6d8 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:43.971753 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dfaa4e-6bea-4ef9-9d18-bbcef6e4d444 None admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:43.972494 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dfaa4e-6bea-4ef9-9d18-bbcef6e4d444 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:58:43.972802 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dfaa4e-6bea-4ef9-9d18-bbcef6e4d444 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:43.974189 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dfaa4e-6bea-4ef9-9d18-bbcef6e4d444 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:43.978338 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 220/447] 10.222.0.23 () {56 vars in 1376 bytes} [Sat Jun 18 04:58:43 2022] GET /identity/v3/role_assignments?group.id=0edcd696603f45549001165885fb7a69&scope.project.id=41cb6d0c536f4c4494fdabc03643aef0&role.id=cbf5c4c26898413d93e73b46b41046a3 => generated 611 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:43.979270 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c84031-cbed-4697-9b86-074b7e76d6d8 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:43.998845 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 228/448] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Jun 18 04:58:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:44.004334 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dfaa4e-6bea-4ef9-9d18-bbcef6e4d444 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:44.095142 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c84031-cbed-4697-9b86-074b7e76d6d8 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:44.126283 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dfaa4e-6bea-4ef9-9d18-bbcef6e4d444 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:44.133255 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2b098b-e860-4906-bb32-f1bc91485c53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:44.134144 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2b098b-e860-4906-bb32-f1bc91485c53 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:44.134432 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2b098b-e860-4906-bb32-f1bc91485c53 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:44.134696 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2b098b-e860-4906-bb32-f1bc91485c53 None admin] PATH_INFO: `/v3/projects/41cb6d0c536f4c4494fdabc03643aef0` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:44.135892 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2b098b-e860-4906-bb32-f1bc91485c53 None admin] RBAC: Authorizing `identity:get_project(project_id=41cb6d0c536f4c4494fdabc03643aef0)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:44.137319 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2b098b-e860-4906-bb32-f1bc91485c53 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:44.139205 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 221/449] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Jun 18 04:58:43 2022] GET /identity/v3/projects/41cb6d0c536f4c4494fdabc03643aef0 => generated 302 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:44.145163 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2b098b-e860-4906-bb32-f1bc91485c53 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:44.171227 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e3398d-1bc1-4719-9d42-423e781fdb1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:44.171883 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e3398d-1bc1-4719-9d42-423e781fdb1f None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:44.172171 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e3398d-1bc1-4719-9d42-423e781fdb1f None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:44.172452 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e3398d-1bc1-4719-9d42-423e781fdb1f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:44.184533 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:44.197883 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:44.248682 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e3398d-1bc1-4719-9d42-423e781fdb1f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:44.250239 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e3398d-1bc1-4719-9d42-423e781fdb1f None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:44.268957 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2b098b-e860-4906-bb32-f1bc91485c53 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:44.310568 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b328ead8-d425-4964-87ab-42ca11ae053f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:44.311419 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b328ead8-d425-4964-87ab-42ca11ae053f None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:44.311699 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b328ead8-d425-4964-87ab-42ca11ae053f None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:44.311976 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b328ead8-d425-4964-87ab-42ca11ae053f None admin] PATH_INFO: `/v3/groups/0edcd696603f45549001165885fb7a69` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:44.315400 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b328ead8-d425-4964-87ab-42ca11ae053f None admin] RBAC: Authorizing `identity:get_group(group_id=0edcd696603f45549001165885fb7a69)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:44.315400 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b328ead8-d425-4964-87ab-42ca11ae053f None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:44.316226 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 222/450] 10.222.0.23 () {56 vars in 1180 bytes} [Sat Jun 18 04:58:44 2022] GET /identity/v3/groups/0edcd696603f45549001165885fb7a69 => generated 226 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:44.322659 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b328ead8-d425-4964-87ab-42ca11ae053f None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:44.366973 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 229/451] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Jun 18 04:58:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 18 04:58:44.372678 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e3398d-1bc1-4719-9d42-423e781fdb1f None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:44.420033 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:44.430680 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b328ead8-d425-4964-87ab-42ca11ae053f None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:44.437723 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:44.464085 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b25d95-39e5-4448-bbf6-fdb76fad4063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:44.464723 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b25d95-39e5-4448-bbf6-fdb76fad4063 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:44.465095 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b25d95-39e5-4448-bbf6-fdb76fad4063 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:44.465320 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b25d95-39e5-4448-bbf6-fdb76fad4063 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:44.466172 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b25d95-39e5-4448-bbf6-fdb76fad4063 None admin] RBAC: Adding query filter params (group.id=0edcd696603f45549001165885fb7a69, role.id=0089a3aad83c4ae08d8e9d864957285b, scope.project.id=41cb6d0c536f4c4494fdabc03643aef0) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:58:44.466480 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b25d95-39e5-4448-bbf6-fdb76fad4063 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:44.467905 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b25d95-39e5-4448-bbf6-fdb76fad4063 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:44.474012 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 223/452] 10.222.0.23 () {56 vars in 1376 bytes} [Sat Jun 18 04:58:44 2022] GET /identity/v3/role_assignments?group.id=0edcd696603f45549001165885fb7a69&scope.project.id=41cb6d0c536f4c4494fdabc03643aef0&role.id=0089a3aad83c4ae08d8e9d864957285b => generated 611 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:44.477464 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e3398d-1bc1-4719-9d42-423e781fdb1f None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:44.515841 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0377bb4c-993c-4115-a007-ea1e39e69169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:44.516624 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0377bb4c-993c-4115-a007-ea1e39e69169 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:44.516904 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0377bb4c-993c-4115-a007-ea1e39e69169 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:44.517174 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0377bb4c-993c-4115-a007-ea1e39e69169 None admin] PATH_INFO: `/v3/projects/d587d7347f6f4de2882df28dc6b6a574` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:44.518439 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0377bb4c-993c-4115-a007-ea1e39e69169 None admin] RBAC: Authorizing `identity:get_project(project_id=d587d7347f6f4de2882df28dc6b6a574)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:44.519946 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0377bb4c-993c-4115-a007-ea1e39e69169 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:44.522122 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 230/453] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Jun 18 04:58:44 2022] GET /identity/v3/projects/d587d7347f6f4de2882df28dc6b6a574 => generated 344 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 18 04:58:44.527664 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b25d95-39e5-4448-bbf6-fdb76fad4063 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:44.634868 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b25d95-39e5-4448-bbf6-fdb76fad4063 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:44.671039 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ead815-3db7-4820-b793-63f38ad4e7be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:44.671893 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ead815-3db7-4820-b793-63f38ad4e7be None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:44.672179 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ead815-3db7-4820-b793-63f38ad4e7be None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:44.672456 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ead815-3db7-4820-b793-63f38ad4e7be None admin] PATH_INFO: `/v3/groups/6e930b1dfac546d6a830d4cc1808c375` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:44.673713 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ead815-3db7-4820-b793-63f38ad4e7be None admin] RBAC: Authorizing `identity:get_group(group_id=6e930b1dfac546d6a830d4cc1808c375)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:44.675146 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ead815-3db7-4820-b793-63f38ad4e7be None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:44.678713 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 224/454] 10.222.0.23 () {56 vars in 1180 bytes} [Sat Jun 18 04:58:44 2022] GET /identity/v3/groups/6e930b1dfac546d6a830d4cc1808c375 => generated 229 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:44.683425 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0377bb4c-993c-4115-a007-ea1e39e69169 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:44.784730 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0377bb4c-993c-4115-a007-ea1e39e69169 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:44.795533 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:44.820902 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0371d0-876a-42f9-91f4-cce32fa086ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:44.821638 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0371d0-876a-42f9-91f4-cce32fa086ea None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:44.822130 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0371d0-876a-42f9-91f4-cce32fa086ea None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:44.823067 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0371d0-876a-42f9-91f4-cce32fa086ea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:44.823899 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0371d0-876a-42f9-91f4-cce32fa086ea None admin] RBAC: Adding query filter params (group.id=6e930b1dfac546d6a830d4cc1808c375, role.id=21c65f2c51bd4dd283d13507d2714467, scope.project.id=d587d7347f6f4de2882df28dc6b6a574) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:58:44.824215 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0371d0-876a-42f9-91f4-cce32fa086ea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:44.825693 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0371d0-876a-42f9-91f4-cce32fa086ea None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:44.831697 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 231/455] 10.222.0.23 () {56 vars in 1376 bytes} [Sat Jun 18 04:58:44 2022] GET /identity/v3/role_assignments?group.id=6e930b1dfac546d6a830d4cc1808c375&scope.project.id=d587d7347f6f4de2882df28dc6b6a574&role.id=21c65f2c51bd4dd283d13507d2714467 => generated 611 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:47.309002 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e639f5ca-384c-4407-bf41-e51bac55ead4 None None] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:47.309786 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e639f5ca-384c-4407-bf41-e51bac55ead4 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:47.309786 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e639f5ca-384c-4407-bf41-e51bac55ead4 None None] PATH_INFO: `/` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:47.311204 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 225/456] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 04:58:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 04:58:47.316753 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf1a8b2-5ff2-4cda-987b-9624bdd7a5d5 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:47.317416 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf1a8b2-5ff2-4cda-987b-9624bdd7a5d5 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:47.317416 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf1a8b2-5ff2-4cda-987b-9624bdd7a5d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:47.342479 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-0bf1a8b2-5ff2-4cda-987b-9624bdd7a5d5 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 04:58:47.343697 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-0bf1a8b2-5ff2-4cda-987b-9624bdd7a5d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 04:58:47.456085 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 232/457] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 04:58:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 18 04:58:47.480808 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e639f5ca-384c-4407-bf41-e51bac55ead4 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:47.539834 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:47.623775 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e639f5ca-384c-4407-bf41-e51bac55ead4 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:47.681971 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7e63d2-15fd-4848-a9ed-0e2f80b143e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:47.683313 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7e63d2-15fd-4848-a9ed-0e2f80b143e0 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:47.683817 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7e63d2-15fd-4848-a9ed-0e2f80b143e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:47.684313 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7e63d2-15fd-4848-a9ed-0e2f80b143e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:47.702935 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1db62f0-47cd-4362-9a2e-f243f03c2269 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:47.703420 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1db62f0-47cd-4362-9a2e-f243f03c2269 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:47.703862 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1db62f0-47cd-4362-9a2e-f243f03c2269 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:47.709400 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 233/458] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 04:58:47 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 04:58:47.710733 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7e63d2-15fd-4848-a9ed-0e2f80b143e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:47.711680 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2282c78-8800-40a6-baf7-180245f8a37d None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:47.712218 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2282c78-8800-40a6-baf7-180245f8a37d None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:47.712711 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2282c78-8800-40a6-baf7-180245f8a37d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:47.713518 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7e63d2-15fd-4848-a9ed-0e2f80b143e0 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:47.731105 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-e2282c78-8800-40a6-baf7-180245f8a37d None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 04:58:47.732563 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-e2282c78-8800-40a6-baf7-180245f8a37d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 04:58:47.751339 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application [None req-2a7e63d2-15fd-4848-a9ed-0e2f80b143e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 04:58:47.751339 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:58:47.751339 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:58:47.751339 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:58:47.751339 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:58:47.751339 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:58:47.751339 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:58:47.751339 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:58:47.751339 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:58:47.751339 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:58:47.751339 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:58:47.751339 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:58:47.751339 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 18 04:58:47.751339 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 18 04:58:47.751339 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 18 04:58:47.752281 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 18 04:58:47.752281 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:58:47.752281 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:58:47.752281 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:58:47.752281 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:58:47.752281 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:58:47.752281 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:58:47.752281 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:58:47.752281 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application with Lock( Jun 18 04:58:47.752281 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:58:47.752281 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:58:47.752281 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:58:47.752281 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:58:47.752281 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:58:47.752281 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:58:47.752281 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:58:47.752281 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:58:47.753115 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 18 04:58:47.753115 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 18 04:58:47.753115 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 04:58:47.753115 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 04:58:47.765612 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 226/459] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Jun 18 04:58:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:58:47.768210 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7e63d2-15fd-4848-a9ed-0e2f80b143e0 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:47.845799 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 234/460] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 04:58:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 18 04:58:47.847075 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18086e19-5c73-4cf1-926a-f2e9abff1559 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:47.848555 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18086e19-5c73-4cf1-926a-f2e9abff1559 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:47.848837 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18086e19-5c73-4cf1-926a-f2e9abff1559 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:47.850941 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 235/461] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 04:58:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 04:58:47.859276 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7e3891-617b-4a2e-baa1-176a0139e2e1 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:47.859782 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7e3891-617b-4a2e-baa1-176a0139e2e1 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:47.860283 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7e3891-617b-4a2e-baa1-176a0139e2e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:47.894172 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-7d7e3891-617b-4a2e-baa1-176a0139e2e1 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 04:58:47.896111 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-7d7e3891-617b-4a2e-baa1-176a0139e2e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 04:58:47.913106 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:47.924631 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7e63d2-15fd-4848-a9ed-0e2f80b143e0 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:47.947041 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:47.966302 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5239a57f-2d2c-4323-a8db-4b36a5941053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:47.967286 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5239a57f-2d2c-4323-a8db-4b36a5941053 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:47.967762 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5239a57f-2d2c-4323-a8db-4b36a5941053 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:47.968237 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5239a57f-2d2c-4323-a8db-4b36a5941053 None admin] PATH_INFO: `/v3/domains` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:47.969155 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5239a57f-2d2c-4323-a8db-4b36a5941053 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:58:47.969688 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5239a57f-2d2c-4323-a8db-4b36a5941053 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:47.971381 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5239a57f-2d2c-4323-a8db-4b36a5941053 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:47.992024 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 227/462] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Jun 18 04:58:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:47.992024 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5239a57f-2d2c-4323-a8db-4b36a5941053 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:48.011967 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 236/463] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 04:58:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 18 04:58:48.012764 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7e3891-617b-4a2e-baa1-176a0139e2e1 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:48.134613 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5239a57f-2d2c-4323-a8db-4b36a5941053 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:48.138727 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7e3891-617b-4a2e-baa1-176a0139e2e1 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:48.178413 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba7090b-7fec-4408-8192-5458663e068e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:48.179578 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba7090b-7fec-4408-8192-5458663e068e None admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:48.179859 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba7090b-7fec-4408-8192-5458663e068e None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:48.180146 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba7090b-7fec-4408-8192-5458663e068e None admin] PATH_INFO: `/v3/users` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:48.180923 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba7090b-7fec-4408-8192-5458663e068e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:48.181136 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992b4ec8-5301-45fe-9710-c5225c616d29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:48.182193 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992b4ec8-5301-45fe-9710-c5225c616d29 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:48.182411 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba7090b-7fec-4408-8192-5458663e068e None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:48.182871 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992b4ec8-5301-45fe-9710-c5225c616d29 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:48.183333 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992b4ec8-5301-45fe-9710-c5225c616d29 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:48.197098 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992b4ec8-5301-45fe-9710-c5225c616d29 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:48.198951 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992b4ec8-5301-45fe-9710-c5225c616d29 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:48.220049 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application [None req-992b4ec8-5301-45fe-9710-c5225c616d29 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 04:58:48.220049 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:58:48.220049 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:58:48.220049 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:58:48.220049 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:58:48.220049 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:58:48.220049 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:58:48.220049 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:58:48.220049 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:58:48.220049 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:58:48.220049 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:58:48.220049 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:58:48.220049 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 18 04:58:48.220049 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 18 04:58:48.220049 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 18 04:58:48.221013 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 18 04:58:48.221013 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:58:48.221013 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:58:48.221013 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:58:48.221013 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:58:48.221013 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:58:48.221013 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:58:48.221013 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:58:48.221013 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application with Lock( Jun 18 04:58:48.221013 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:58:48.221013 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:58:48.221013 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:58:48.221013 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:58:48.221013 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:58:48.221013 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:58:48.221013 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:58:48.221013 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:58:48.221873 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 18 04:58:48.221873 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 18 04:58:48.221873 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 04:58:48.221873 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 04:58:48.228380 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 228/464] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Jun 18 04:58:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:58:48.228380 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992b4ec8-5301-45fe-9710-c5225c616d29 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:48.236034 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:48.301406 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 237/465] 10.222.0.23 () {60 vars in 1132 bytes} [Sat Jun 18 04:58:48 2022] POST /identity/v3/users => generated 246 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 04:58:48.301406 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba7090b-7fec-4408-8192-5458663e068e None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:48.341571 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:48.410019 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992b4ec8-5301-45fe-9710-c5225c616d29 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:48.455176 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e792ff9e-06c7-48ef-8a6a-b1e44d424f2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:48.455888 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e792ff9e-06c7-48ef-8a6a-b1e44d424f2a None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:48.456168 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e792ff9e-06c7-48ef-8a6a-b1e44d424f2a None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:48.456440 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e792ff9e-06c7-48ef-8a6a-b1e44d424f2a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:48.472163 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e792ff9e-06c7-48ef-8a6a-b1e44d424f2a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:48.473692 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e792ff9e-06c7-48ef-8a6a-b1e44d424f2a None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:48.503768 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application [None req-e792ff9e-06c7-48ef-8a6a-b1e44d424f2a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 04:58:48.503768 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:58:48.503768 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:58:48.503768 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:58:48.503768 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:58:48.503768 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:58:48.503768 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:58:48.503768 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:58:48.503768 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:58:48.503768 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:58:48.503768 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:58:48.503768 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:58:48.503768 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 18 04:58:48.503768 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 18 04:58:48.503768 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 18 04:58:48.504682 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 18 04:58:48.504682 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:58:48.504682 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:58:48.504682 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:58:48.504682 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:58:48.504682 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:58:48.504682 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:58:48.504682 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:58:48.504682 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application with Lock( Jun 18 04:58:48.504682 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:58:48.504682 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:58:48.504682 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:58:48.504682 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:58:48.504682 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:58:48.504682 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:58:48.504682 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:58:48.504682 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:58:48.505553 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 18 04:58:48.505553 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 18 04:58:48.505553 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 04:58:48.505553 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 04:58:48.508408 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 229/466] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Jun 18 04:58:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:58:48.516962 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e792ff9e-06c7-48ef-8a6a-b1e44d424f2a None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:48.525446 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba7090b-7fec-4408-8192-5458663e068e None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:48.565160 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb4215c-518d-46ab-bc8f-81bcaa188a1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:48.565876 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb4215c-518d-46ab-bc8f-81bcaa188a1a None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:48.566163 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb4215c-518d-46ab-bc8f-81bcaa188a1a None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:48.566485 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb4215c-518d-46ab-bc8f-81bcaa188a1a None admin] PATH_INFO: `/v3/domains` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:48.567218 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb4215c-518d-46ab-bc8f-81bcaa188a1a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:58:48.567634 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb4215c-518d-46ab-bc8f-81bcaa188a1a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:48.569230 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb4215c-518d-46ab-bc8f-81bcaa188a1a None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:48.583203 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 238/467] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Jun 18 04:58:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:48.584507 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:48.588895 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb4215c-518d-46ab-bc8f-81bcaa188a1a None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:48.634536 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e792ff9e-06c7-48ef-8a6a-b1e44d424f2a None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:48.672615 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dde44b-14a4-4d97-8d2e-3351381bdbe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:48.672615 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dde44b-14a4-4d97-8d2e-3351381bdbe1 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:48.672880 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dde44b-14a4-4d97-8d2e-3351381bdbe1 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:48.673123 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dde44b-14a4-4d97-8d2e-3351381bdbe1 None admin] PATH_INFO: `/v3/domains` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:48.673761 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68dde44b-14a4-4d97-8d2e-3351381bdbe1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:58:48.674077 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68dde44b-14a4-4d97-8d2e-3351381bdbe1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:48.675473 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68dde44b-14a4-4d97-8d2e-3351381bdbe1 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:48.683475 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:48.694255 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 230/468] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Jun 18 04:58:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:48.697641 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dde44b-14a4-4d97-8d2e-3351381bdbe1 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:48.718516 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb4215c-518d-46ab-bc8f-81bcaa188a1a None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:48.767665 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172dca2e-8794-406a-9f8c-91e72d3869ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:48.768720 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172dca2e-8794-406a-9f8c-91e72d3869ff None admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:48.769299 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172dca2e-8794-406a-9f8c-91e72d3869ff None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:48.769808 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172dca2e-8794-406a-9f8c-91e72d3869ff None admin] PATH_INFO: `/v3/users` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:48.770723 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172dca2e-8794-406a-9f8c-91e72d3869ff None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:48.773190 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172dca2e-8794-406a-9f8c-91e72d3869ff None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:48.820341 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dde44b-14a4-4d97-8d2e-3351381bdbe1 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:48.836888 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:48.857048 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 239/469] 10.222.0.23 () {60 vars in 1132 bytes} [Sat Jun 18 04:58:48 2022] POST /identity/v3/users => generated 248 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 04:58:48.858271 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dbced6-74ab-4a53-87d6-d97649ac0776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:48.859041 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dbced6-74ab-4a53-87d6-d97649ac0776 None admin] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:48.859323 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dbced6-74ab-4a53-87d6-d97649ac0776 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:48.859569 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734913a4-9159-48b1-af00-777ba7122c91 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:48.859671 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dbced6-74ab-4a53-87d6-d97649ac0776 None admin] PATH_INFO: `/v3/users` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:48.860155 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734913a4-9159-48b1-af00-777ba7122c91 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:48.860419 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dbced6-74ab-4a53-87d6-d97649ac0776 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:48.860876 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734913a4-9159-48b1-af00-777ba7122c91 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:48.861997 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dbced6-74ab-4a53-87d6-d97649ac0776 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:48.863059 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 240/470] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 04:58:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 04:58:48.869662 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3400b3b-28f5-4e50-91e7-61537d027ad4 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:48.870575 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3400b3b-28f5-4e50-91e7-61537d027ad4 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:48.871061 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3400b3b-28f5-4e50-91e7-61537d027ad4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:48.894781 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-b3400b3b-28f5-4e50-91e7-61537d027ad4 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 04:58:48.896972 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-b3400b3b-28f5-4e50-91e7-61537d027ad4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 04:58:48.973843 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 231/471] 10.222.0.23 () {60 vars in 1132 bytes} [Sat Jun 18 04:58:48 2022] POST /identity/v3/users => generated 248 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 04:58:48.998049 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 241/472] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 04:58:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 18 04:58:49.040428 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dbced6-74ab-4a53-87d6-d97649ac0776 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:49.158753 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dbced6-74ab-4a53-87d6-d97649ac0776 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:49.197515 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0650fa4c-96c7-4d67-99f0-4e1bf08b9c09 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:49.198216 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0650fa4c-96c7-4d67-99f0-4e1bf08b9c09 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:49.198783 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0650fa4c-96c7-4d67-99f0-4e1bf08b9c09 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:49.200746 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961d9232-96cf-4484-a020-73d5ef72c43a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:49.201183 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 242/473] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 04:58:49 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 04:58:49.203456 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961d9232-96cf-4484-a020-73d5ef72c43a None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:49.203456 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961d9232-96cf-4484-a020-73d5ef72c43a None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:49.203456 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961d9232-96cf-4484-a020-73d5ef72c43a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:49.207290 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8698d1c1-28c7-4b64-9d40-1b9f1807abd6 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:49.207290 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8698d1c1-28c7-4b64-9d40-1b9f1807abd6 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:49.207500 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8698d1c1-28c7-4b64-9d40-1b9f1807abd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:49.219859 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961d9232-96cf-4484-a020-73d5ef72c43a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:49.221347 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961d9232-96cf-4484-a020-73d5ef72c43a None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:49.235618 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-8698d1c1-28c7-4b64-9d40-1b9f1807abd6 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 04:58:49.236740 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-8698d1c1-28c7-4b64-9d40-1b9f1807abd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 04:58:49.246058 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application [None req-961d9232-96cf-4484-a020-73d5ef72c43a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 04:58:49.246058 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:58:49.246058 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:58:49.246058 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:58:49.246058 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:58:49.246058 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:58:49.246058 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:58:49.246058 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:58:49.246058 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:58:49.246058 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:58:49.246058 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:58:49.246058 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:58:49.246058 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 18 04:58:49.246058 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 18 04:58:49.246058 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 18 04:58:49.246972 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 18 04:58:49.246972 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:58:49.246972 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:58:49.246972 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:58:49.246972 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:58:49.246972 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:58:49.246972 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:58:49.246972 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:58:49.246972 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application with Lock( Jun 18 04:58:49.246972 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:58:49.246972 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:58:49.246972 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:58:49.246972 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:58:49.246972 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:58:49.246972 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:58:49.246972 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:58:49.246972 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:58:49.247839 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 18 04:58:49.247839 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 18 04:58:49.247839 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 04:58:49.247839 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 04:58:49.251557 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 232/474] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Jun 18 04:58:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:58:49.260802 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961d9232-96cf-4484-a020-73d5ef72c43a None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:49.297738 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:49.313064 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:49.325396 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 243/475] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 04:58:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 18 04:58:49.356544 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8698d1c1-28c7-4b64-9d40-1b9f1807abd6 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:49.395278 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961d9232-96cf-4484-a020-73d5ef72c43a None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:49.434074 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86057eec-ad3e-4548-a520-6807331f3fc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:49.434723 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86057eec-ad3e-4548-a520-6807331f3fc0 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:49.435025 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86057eec-ad3e-4548-a520-6807331f3fc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:49.435297 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86057eec-ad3e-4548-a520-6807331f3fc0 None admin] PATH_INFO: `/v3/domains` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:49.436043 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86057eec-ad3e-4548-a520-6807331f3fc0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:58:49.436364 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86057eec-ad3e-4548-a520-6807331f3fc0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:49.437797 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86057eec-ad3e-4548-a520-6807331f3fc0 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:49.452960 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 233/476] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Jun 18 04:58:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 18 04:58:49.458239 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86057eec-ad3e-4548-a520-6807331f3fc0 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:49.484078 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8698d1c1-28c7-4b64-9d40-1b9f1807abd6 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:49.522694 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e33025-3185-475f-a2e0-f1c5096a2f31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:49.523614 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e33025-3185-475f-a2e0-f1c5096a2f31 None admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:49.524041 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e33025-3185-475f-a2e0-f1c5096a2f31 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:49.524496 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e33025-3185-475f-a2e0-f1c5096a2f31 None admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:49.525456 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e33025-3185-475f-a2e0-f1c5096a2f31 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:49.527008 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e33025-3185-475f-a2e0-f1c5096a2f31 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:49.556008 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:49.564924 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.sql.core [None req-07e33025-3185-475f-a2e0-f1c5096a2f31 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 18 04:58:49.564924 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 18 04:58:49.564924 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [parameters: {'id': 'a4ff3eae857a4dff8dd20a258513a3fc', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 18 04:58:49.564924 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80469) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 18 04:58:49.568341 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86057eec-ad3e-4548-a520-6807331f3fc0 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:49.590619 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-07e33025-3185-475f-a2e0-f1c5096a2f31 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 18 04:58:49.590619 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:58:49.590619 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 18 04:58:49.590619 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 18 04:58:49.590619 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 18 04:58:49.590619 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 18 04:58:49.590619 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 18 04:58:49.590619 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application result = self._query(query) Jun 18 04:58:49.590619 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 18 04:58:49.590619 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application conn.query(q) Jun 18 04:58:49.590619 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 18 04:58:49.590619 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 18 04:58:49.590619 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 18 04:58:49.590619 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application result.read() Jun 18 04:58:49.590619 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 18 04:58:49.591776 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 18 04:58:49.591776 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 18 04:58:49.591776 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 18 04:58:49.591776 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 18 04:58:49.591776 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 18 04:58:49.591776 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 18 04:58:49.591776 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 18 04:58:49.591776 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 18 04:58:49.591776 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 04:58:49.591776 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 18 04:58:49.591776 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 04:58:49.591776 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:58:49.591776 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 18 04:58:49.591776 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 18 04:58:49.591776 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jun 18 04:58:49.591776 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return ref.to_dict() Jun 18 04:58:49.591776 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 18 04:58:49.591776 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application next(self.gen) Jun 18 04:58:49.592901 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 18 04:58:49.592901 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application yield resource Jun 18 04:58:49.592901 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 18 04:58:49.592901 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application next(self.gen) Jun 18 04:58:49.592901 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 18 04:58:49.592901 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application self.session.rollback() Jun 18 04:58:49.592901 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 18 04:58:49.592901 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application self.force_reraise() Jun 18 04:58:49.592901 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 18 04:58:49.592901 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise self.value Jun 18 04:58:49.592901 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 18 04:58:49.592901 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 18 04:58:49.592901 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 18 04:58:49.592901 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application session.commit() Jun 18 04:58:49.592901 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 18 04:58:49.592901 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 18 04:58:49.592901 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 18 04:58:49.594536 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application self._prepare_impl() Jun 18 04:58:49.594536 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 18 04:58:49.594536 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application self.session.flush() Jun 18 04:58:49.594536 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 18 04:58:49.594536 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application self._flush(objects) Jun 18 04:58:49.594536 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 18 04:58:49.594536 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 18 04:58:49.594536 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 18 04:58:49.594536 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application compat.raise_( Jun 18 04:58:49.594536 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 18 04:58:49.594536 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception Jun 18 04:58:49.594536 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 18 04:58:49.594536 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application flush_context.execute() Jun 18 04:58:49.594536 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 18 04:58:49.594536 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rec.execute(self) Jun 18 04:58:49.594536 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 18 04:58:49.594536 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 18 04:58:49.596036 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 18 04:58:49.596036 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 18 04:58:49.596036 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 18 04:58:49.596036 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 18 04:58:49.596036 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 18 04:58:49.596036 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 18 04:58:49.596036 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 18 04:58:49.596036 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 18 04:58:49.596036 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 18 04:58:49.596036 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 18 04:58:49.596036 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 18 04:58:49.596036 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 18 04:58:49.596036 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 18 04:58:49.596036 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 18 04:58:49.596036 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 18 04:58:49.596036 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception Jun 18 04:58:49.597338 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 18 04:58:49.597338 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 18 04:58:49.597338 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 18 04:58:49.597338 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 18 04:58:49.597338 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 18 04:58:49.597338 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application result = self._query(query) Jun 18 04:58:49.597338 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 18 04:58:49.597338 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application conn.query(q) Jun 18 04:58:49.597338 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 18 04:58:49.597338 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 18 04:58:49.597338 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 18 04:58:49.597338 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application result.read() Jun 18 04:58:49.597338 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 18 04:58:49.597338 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 18 04:58:49.597338 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 18 04:58:49.597338 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 18 04:58:49.601033 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 18 04:58:49.601033 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 18 04:58:49.601033 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 18 04:58:49.601033 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 18 04:58:49.601033 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 18 04:58:49.601033 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 18 04:58:49.601033 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [parameters: {'id': 'a4ff3eae857a4dff8dd20a258513a3fc', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 18 04:58:49.601033 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 18 04:58:49.601033 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 04:58:49.601033 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 18 04:58:49.601033 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 04:58:49.601033 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:58:49.601033 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:58:49.601033 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:58:49.601033 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:58:49.601033 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:58:49.602550 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:58:49.602550 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:58:49.602550 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:58:49.602550 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:58:49.602550 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:58:49.602550 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:58:49.602550 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jun 18 04:58:49.602550 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jun 18 04:58:49.602550 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:58:49.602550 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:58:49.602550 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jun 18 04:58:49.602550 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jun 18 04:58:49.602550 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 18 04:58:49.602550 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 18 04:58:49.602550 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 18 04:58:49.602550 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 04:58:49.605708 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 244/477] 10.222.0.23 () {60 vars in 1131 bytes} [Sat Jun 18 04:58:49 2022] POST /identity/v3/roles => generated 144 bytes in 250 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jun 18 04:58:49.612489 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e33025-3185-475f-a2e0-f1c5096a2f31 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:49.623812 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4974bbc5-c0d0-447a-9cd5-58ccc139b2e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:49.625001 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4974bbc5-c0d0-447a-9cd5-58ccc139b2e8 None admin] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:49.627752 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4974bbc5-c0d0-447a-9cd5-58ccc139b2e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:49.628290 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4974bbc5-c0d0-447a-9cd5-58ccc139b2e8 None admin] PATH_INFO: `/v3/users` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:49.629258 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4974bbc5-c0d0-447a-9cd5-58ccc139b2e8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:49.633686 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4974bbc5-c0d0-447a-9cd5-58ccc139b2e8 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:49.699221 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:49.725855 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 234/478] 10.222.0.23 () {60 vars in 1132 bytes} [Sat Jun 18 04:58:49 2022] POST /identity/v3/users => generated 249 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 04:58:49.766022 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e33025-3185-475f-a2e0-f1c5096a2f31 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:49.852550 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36672d4c-cbbd-40b8-88b0-a51e21e62caa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:49.853307 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36672d4c-cbbd-40b8-88b0-a51e21e62caa None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:49.853638 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36672d4c-cbbd-40b8-88b0-a51e21e62caa None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:49.853913 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36672d4c-cbbd-40b8-88b0-a51e21e62caa None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:49.876097 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36672d4c-cbbd-40b8-88b0-a51e21e62caa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 18 04:58:49.884796 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-36672d4c-cbbd-40b8-88b0-a51e21e62caa None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 18 04:58:49.884796 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:58:49.884796 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:58:49.884796 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:58:49.884796 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:58:49.884796 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:58:49.884796 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:58:49.884796 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:58:49.884796 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:58:49.884796 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:58:49.884796 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:58:49.884796 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:58:49.884796 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 18 04:58:49.884796 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 18 04:58:49.884796 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 18 04:58:49.884796 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 18 04:58:49.885798 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 18 04:58:49.885798 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 18 04:58:49.885798 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 18 04:58:49.885798 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 18 04:58:49.885798 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:58:49.885798 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:58:49.885798 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:58:49.885798 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:58:49.885798 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:58:49.885798 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:58:49.885798 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:58:49.885798 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jun 18 04:58:49.885798 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:58:49.885798 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:58:49.885798 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:58:49.885798 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:58:49.886592 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:58:49.886592 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:58:49.886592 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:58:49.886592 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:58:49.886592 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 18 04:58:49.886592 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 18 04:58:49.886592 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 18 04:58:49.886592 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 18 04:58:49.886592 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 18 04:58:49.886592 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 18 04:58:49.886592 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 18 04:58:49.886592 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 04:58:49.886592 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 245/479] 10.222.0.23 () {56 vars in 1114 bytes} [Sat Jun 18 04:58:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:58:49.893552 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4974bbc5-c0d0-447a-9cd5-58ccc139b2e8 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:50.018445 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4974bbc5-c0d0-447a-9cd5-58ccc139b2e8 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:50.056945 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8733c2-183a-4191-a260-288be796da81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:50.058011 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8733c2-183a-4191-a260-288be796da81 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:50.058526 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8733c2-183a-4191-a260-288be796da81 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:50.059022 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8733c2-183a-4191-a260-288be796da81 None admin] PATH_INFO: `/v3/roles` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:50.059969 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8733c2-183a-4191-a260-288be796da81 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:58:50.060481 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8733c2-183a-4191-a260-288be796da81 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:50.062089 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8733c2-183a-4191-a260-288be796da81 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:50.067581 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:50.073201 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 235/480] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Jun 18 04:58:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:50.962795 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170fb783-c401-45f3-bc51-9b796b77e663 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:50.963299 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170fb783-c401-45f3-bc51-9b796b77e663 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:50.963375 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170fb783-c401-45f3-bc51-9b796b77e663 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:50.965015 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 246/481] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 04:58:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 04:58:50.969921 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cff53db-d961-4194-929f-b413d05819a9 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:50.970727 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cff53db-d961-4194-929f-b413d05819a9 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:50.971213 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cff53db-d961-4194-929f-b413d05819a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:50.994284 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-2cff53db-d961-4194-929f-b413d05819a9 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 04:58:50.995824 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-2cff53db-d961-4194-929f-b413d05819a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 04:58:51.084530 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 236/482] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 04:58:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 18 04:58:51.120865 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170fb783-c401-45f3-bc51-9b796b77e663 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:51.263174 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170fb783-c401-45f3-bc51-9b796b77e663 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:51.286165 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:51.308821 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d184eabc-68cf-4457-8fb1-33de49b30326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:51.309864 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d184eabc-68cf-4457-8fb1-33de49b30326 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:51.310032 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d184eabc-68cf-4457-8fb1-33de49b30326 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:51.310261 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d184eabc-68cf-4457-8fb1-33de49b30326 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:51.325580 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d184eabc-68cf-4457-8fb1-33de49b30326 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 18 04:58:51.337999 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-d184eabc-68cf-4457-8fb1-33de49b30326 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 18 04:58:51.337999 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:58:51.337999 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:58:51.337999 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:58:51.337999 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:58:51.337999 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:58:51.337999 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:58:51.337999 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:58:51.337999 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:58:51.337999 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:58:51.337999 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:58:51.337999 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:58:51.337999 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 18 04:58:51.337999 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 18 04:58:51.337999 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 18 04:58:51.337999 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 18 04:58:51.338921 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 18 04:58:51.338921 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 18 04:58:51.338921 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 18 04:58:51.338921 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 18 04:58:51.338921 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:58:51.338921 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:58:51.338921 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:58:51.338921 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:58:51.338921 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:58:51.338921 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:58:51.338921 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:58:51.338921 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jun 18 04:58:51.338921 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:58:51.338921 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:58:51.338921 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:58:51.338921 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:58:51.339712 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:58:51.339712 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:58:51.339712 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:58:51.339712 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:58:51.339712 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 18 04:58:51.339712 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 18 04:58:51.339712 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 18 04:58:51.339712 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 18 04:58:51.339712 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 18 04:58:51.339712 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 18 04:58:51.339712 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 18 04:58:51.339712 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 04:58:51.339712 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 247/483] 10.222.0.23 () {56 vars in 1102 bytes} [Sat Jun 18 04:58:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:58:51.345112 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cff53db-d961-4194-929f-b413d05819a9 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:51.469491 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fadbcda-e429-4415-aaf9-eb727f1b52fe None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:51.469831 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fadbcda-e429-4415-aaf9-eb727f1b52fe None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:51.470052 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fadbcda-e429-4415-aaf9-eb727f1b52fe None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:51.472768 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 248/484] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 04:58:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 04:58:51.477377 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90f192f-8977-42ce-94dd-eb7585b2aba4 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:51.477690 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90f192f-8977-42ce-94dd-eb7585b2aba4 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:51.478057 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90f192f-8977-42ce-94dd-eb7585b2aba4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:51.485190 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cff53db-d961-4194-929f-b413d05819a9 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:51.517330 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-d90f192f-8977-42ce-94dd-eb7585b2aba4 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 04:58:51.518845 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-d90f192f-8977-42ce-94dd-eb7585b2aba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 04:58:51.528961 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257fa196-1935-40d5-baa6-c91b0503e81a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:51.529726 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257fa196-1935-40d5-baa6-c91b0503e81a None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:51.530016 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257fa196-1935-40d5-baa6-c91b0503e81a None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:51.530292 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257fa196-1935-40d5-baa6-c91b0503e81a None admin] PATH_INFO: `/v3/roles` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:51.531044 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257fa196-1935-40d5-baa6-c91b0503e81a None admin] RBAC: Adding query filter params (name=service) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:58:51.531356 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257fa196-1935-40d5-baa6-c91b0503e81a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:51.532816 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257fa196-1935-40d5-baa6-c91b0503e81a None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:51.542064 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 237/485] 10.222.0.23 () {56 vars in 1103 bytes} [Sat Jun 18 04:58:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:51.542326 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:51.550078 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257fa196-1935-40d5-baa6-c91b0503e81a None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:51.645942 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 249/486] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 04:58:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 18 04:58:51.647426 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f95df7-ee7a-4dcb-b388-78a17f0e8e6e None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:51.647714 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f95df7-ee7a-4dcb-b388-78a17f0e8e6e None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:51.647992 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f95df7-ee7a-4dcb-b388-78a17f0e8e6e None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:51.653178 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 250/487] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 04:58:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 04:58:51.654974 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc510c6-f01e-47fd-acbf-f4d3137bc495 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:51.655255 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc510c6-f01e-47fd-acbf-f4d3137bc495 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:51.655528 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc510c6-f01e-47fd-acbf-f4d3137bc495 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:51.678374 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-bcc510c6-f01e-47fd-acbf-f4d3137bc495 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 04:58:51.679998 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-bcc510c6-f01e-47fd-acbf-f4d3137bc495 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 04:58:51.703905 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257fa196-1935-40d5-baa6-c91b0503e81a None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:51.742157 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee3a50d-101d-4715-8db4-9d889ee5545c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:51.742953 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee3a50d-101d-4715-8db4-9d889ee5545c None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:51.743237 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee3a50d-101d-4715-8db4-9d889ee5545c None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:51.743516 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee3a50d-101d-4715-8db4-9d889ee5545c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:51.755354 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee3a50d-101d-4715-8db4-9d889ee5545c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:51.756927 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee3a50d-101d-4715-8db4-9d889ee5545c None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:51.761413 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 251/488] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 04:58:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 18 04:58:51.762593 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc510c6-f01e-47fd-acbf-f4d3137bc495 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:51.775533 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application [None req-1ee3a50d-101d-4715-8db4-9d889ee5545c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 04:58:51.775533 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:58:51.775533 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:58:51.775533 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:58:51.775533 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:58:51.775533 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:58:51.775533 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:58:51.775533 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:58:51.775533 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:58:51.775533 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:58:51.775533 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:58:51.775533 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:58:51.775533 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 18 04:58:51.775533 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 18 04:58:51.775533 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 18 04:58:51.776370 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 18 04:58:51.776370 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:58:51.776370 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:58:51.776370 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:58:51.776370 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:58:51.776370 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:58:51.776370 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:58:51.776370 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:58:51.776370 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application with Lock( Jun 18 04:58:51.776370 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:58:51.776370 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:58:51.776370 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:58:51.776370 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:58:51.776370 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:58:51.776370 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:58:51.776370 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:58:51.776370 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:58:51.777208 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 18 04:58:51.777208 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 18 04:58:51.777208 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 04:58:51.777208 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 04:58:51.777208 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 238/489] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Jun 18 04:58:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:58:51.782437 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee3a50d-101d-4715-8db4-9d889ee5545c None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:51.831541 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:51.838277 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:51.895662 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee3a50d-101d-4715-8db4-9d889ee5545c None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:51.897451 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc510c6-f01e-47fd-acbf-f4d3137bc495 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:51.935471 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dbdbdb-3817-4715-9357-d77ff364a154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:51.936591 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dbdbdb-3817-4715-9357-d77ff364a154 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:51.936993 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dbdbdb-3817-4715-9357-d77ff364a154 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:51.937488 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dbdbdb-3817-4715-9357-d77ff364a154 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:51.938963 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2fbcdf-c7a4-4cbe-960a-6045069a0fff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:51.940112 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2fbcdf-c7a4-4cbe-960a-6045069a0fff None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:51.940673 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2fbcdf-c7a4-4cbe-960a-6045069a0fff None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:51.941246 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2fbcdf-c7a4-4cbe-960a-6045069a0fff None admin] PATH_INFO: `/v3/domains` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:51.942149 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2fbcdf-c7a4-4cbe-960a-6045069a0fff None admin] RBAC: Adding query filter params (name=Default) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:58:51.942598 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2fbcdf-c7a4-4cbe-960a-6045069a0fff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:51.944694 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2fbcdf-c7a4-4cbe-960a-6045069a0fff None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:51.949817 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51dbdbdb-3817-4715-9357-d77ff364a154 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 18 04:58:51.957813 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-51dbdbdb-3817-4715-9357-d77ff364a154 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 18 04:58:51.957813 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:58:51.957813 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:58:51.957813 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:58:51.957813 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:58:51.957813 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:58:51.957813 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:58:51.957813 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:58:51.957813 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:58:51.957813 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:58:51.957813 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:58:51.957813 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:58:51.957813 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 18 04:58:51.957813 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 18 04:58:51.957813 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 18 04:58:51.957813 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 18 04:58:51.958813 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 18 04:58:51.958813 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 18 04:58:51.958813 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 18 04:58:51.958813 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 18 04:58:51.958813 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:58:51.958813 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:58:51.958813 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:58:51.958813 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:58:51.958813 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:58:51.958813 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:58:51.958813 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:58:51.958813 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jun 18 04:58:51.958813 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:58:51.958813 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:58:51.958813 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:58:51.958813 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:58:51.959606 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:58:51.959606 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:58:51.959606 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:58:51.959606 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:58:51.959606 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 18 04:58:51.959606 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 18 04:58:51.959606 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 18 04:58:51.959606 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 18 04:58:51.959606 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 18 04:58:51.959606 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 18 04:58:51.959606 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 18 04:58:51.959606 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 04:58:51.960197 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 239/490] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Jun 18 04:58:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:51.960279 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 252/491] 10.222.0.23 () {56 vars in 1102 bytes} [Sat Jun 18 04:58:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:58:51.961527 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dbdbdb-3817-4715-9357-d77ff364a154 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:51.966103 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2fbcdf-c7a4-4cbe-960a-6045069a0fff None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:52.066439 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:52.071885 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2fbcdf-c7a4-4cbe-960a-6045069a0fff None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:52.074135 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dbdbdb-3817-4715-9357-d77ff364a154 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:52.074522 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:52.110906 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8c72c1-bccd-4864-91e0-12ecb54f1a5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:52.111488 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235c262a-a54f-491f-b738-c9a6b093a1c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:52.112100 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8c72c1-bccd-4864-91e0-12ecb54f1a5d None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:52.112336 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235c262a-a54f-491f-b738-c9a6b093a1c0 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:52.112723 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8c72c1-bccd-4864-91e0-12ecb54f1a5d None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:52.112830 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235c262a-a54f-491f-b738-c9a6b093a1c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:52.113164 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235c262a-a54f-491f-b738-c9a6b093a1c0 None admin] PATH_INFO: `/v3/roles` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:52.113356 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8c72c1-bccd-4864-91e0-12ecb54f1a5d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:52.114125 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235c262a-a54f-491f-b738-c9a6b093a1c0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:58:52.114535 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235c262a-a54f-491f-b738-c9a6b093a1c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:52.117065 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235c262a-a54f-491f-b738-c9a6b093a1c0 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:52.126164 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8c72c1-bccd-4864-91e0-12ecb54f1a5d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 18 04:58:52.129917 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 240/492] 10.222.0.23 () {56 vars in 1103 bytes} [Sat Jun 18 04:58:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:52.130853 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235c262a-a54f-491f-b738-c9a6b093a1c0 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:52.135509 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-8f8c72c1-bccd-4864-91e0-12ecb54f1a5d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 18 04:58:52.135509 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:58:52.135509 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:58:52.135509 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:58:52.135509 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:58:52.135509 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:58:52.135509 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:58:52.135509 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:58:52.135509 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:58:52.135509 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:58:52.135509 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:58:52.135509 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:58:52.135509 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 18 04:58:52.135509 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 18 04:58:52.135509 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 18 04:58:52.135509 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 18 04:58:52.136391 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 18 04:58:52.136391 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 18 04:58:52.136391 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 18 04:58:52.136391 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 18 04:58:52.136391 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:58:52.136391 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:58:52.136391 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:58:52.136391 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:58:52.136391 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:58:52.136391 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:58:52.136391 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:58:52.136391 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jun 18 04:58:52.136391 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:58:52.136391 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:58:52.136391 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:58:52.136391 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:58:52.137180 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:58:52.137180 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:58:52.137180 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:58:52.137180 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:58:52.137180 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 18 04:58:52.137180 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 18 04:58:52.137180 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 18 04:58:52.137180 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 18 04:58:52.137180 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 18 04:58:52.137180 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 18 04:58:52.137180 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 18 04:58:52.137180 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 04:58:52.139672 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 253/493] 10.222.0.23 () {56 vars in 1102 bytes} [Sat Jun 18 04:58:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:58:52.140688 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8c72c1-bccd-4864-91e0-12ecb54f1a5d None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:52.248531 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8c72c1-bccd-4864-91e0-12ecb54f1a5d None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:52.249616 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235c262a-a54f-491f-b738-c9a6b093a1c0 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:52.286283 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51126e42-ce67-4d96-85fd-6b3f3604e9b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:52.287379 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa817275-f46f-466c-bb05-4633a79433cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:52.288164 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51126e42-ce67-4d96-85fd-6b3f3604e9b0 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:52.288390 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa817275-f46f-466c-bb05-4633a79433cb None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:52.288742 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51126e42-ce67-4d96-85fd-6b3f3604e9b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:52.288849 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa817275-f46f-466c-bb05-4633a79433cb None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:52.289198 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa817275-f46f-466c-bb05-4633a79433cb None admin] PATH_INFO: `/v3/users/nova` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:52.289372 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51126e42-ce67-4d96-85fd-6b3f3604e9b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:52.297808 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa817275-f46f-466c-bb05-4633a79433cb None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:52.299496 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa817275-f46f-466c-bb05-4633a79433cb None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:52.301563 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51126e42-ce67-4d96-85fd-6b3f3604e9b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:52.303913 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51126e42-ce67-4d96-85fd-6b3f3604e9b0 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:52.315993 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application [None req-fa817275-f46f-466c-bb05-4633a79433cb None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 18 04:58:52.315993 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:58:52.315993 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:58:52.315993 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:58:52.315993 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:58:52.315993 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:58:52.315993 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:58:52.315993 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:58:52.315993 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:58:52.315993 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:58:52.315993 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:58:52.315993 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:58:52.315993 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 18 04:58:52.315993 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 18 04:58:52.315993 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 18 04:58:52.315993 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 18 04:58:52.316965 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:58:52.316965 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:58:52.316965 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 18 04:58:52.316965 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 18 04:58:52.316965 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 18 04:58:52.316965 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 18 04:58:52.316965 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:58:52.316965 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:58:52.316965 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:58:52.316965 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:58:52.316965 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:58:52.316965 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application with Lock( Jun 18 04:58:52.316965 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:58:52.316965 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:58:52.316965 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:58:52.316965 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:58:52.316965 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:58:52.316965 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:58:52.317930 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:58:52.317930 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:58:52.317930 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 18 04:58:52.317930 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 18 04:58:52.317930 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 18 04:58:52.317930 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 18 04:58:52.317930 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 18 04:58:52.317930 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 18 04:58:52.317930 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 18 04:58:52.317930 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 04:58:52.322481 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 241/494] 10.222.0.23 () {56 vars in 1093 bytes} [Sat Jun 18 04:58:52 2022] GET /identity/v3/users/nova => generated 82 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:58:52.323351 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa817275-f46f-466c-bb05-4633a79433cb None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:52.326443 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-51126e42-ce67-4d96-85fd-6b3f3604e9b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 04:58:52.326443 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:58:52.326443 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:58:52.326443 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:58:52.326443 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:58:52.326443 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:58:52.326443 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:58:52.326443 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:58:52.326443 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:58:52.326443 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:58:52.326443 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:58:52.326443 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:58:52.326443 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 18 04:58:52.326443 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 18 04:58:52.326443 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 18 04:58:52.327270 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 18 04:58:52.327270 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:58:52.327270 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:58:52.327270 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:58:52.327270 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:58:52.327270 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:58:52.327270 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:58:52.327270 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:58:52.327270 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jun 18 04:58:52.327270 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:58:52.327270 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:58:52.327270 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:58:52.327270 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:58:52.327270 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:58:52.327270 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:58:52.327270 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:58:52.327270 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:58:52.328102 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 18 04:58:52.328102 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 18 04:58:52.328102 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 04:58:52.328102 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 04:58:52.330338 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 254/495] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Jun 18 04:58:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:58:52.331422 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51126e42-ce67-4d96-85fd-6b3f3604e9b0 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:52.346266 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:52.355624 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:52.439675 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa817275-f46f-466c-bb05-4633a79433cb None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:52.449449 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51126e42-ce67-4d96-85fd-6b3f3604e9b0 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:52.475180 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3732185-ece0-447e-b0da-5cf03c97af44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:52.475543 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3732185-ece0-447e-b0da-5cf03c97af44 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:52.475830 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3732185-ece0-447e-b0da-5cf03c97af44 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:52.476114 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3732185-ece0-447e-b0da-5cf03c97af44 None admin] PATH_INFO: `/v3/roles` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:52.476866 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3732185-ece0-447e-b0da-5cf03c97af44 None admin] RBAC: Adding query filter params (name=service) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:58:52.477181 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3732185-ece0-447e-b0da-5cf03c97af44 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:52.478594 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3732185-ece0-447e-b0da-5cf03c97af44 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:52.490742 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 242/496] 10.222.0.23 () {56 vars in 1103 bytes} [Sat Jun 18 04:58:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:52.490742 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3732185-ece0-447e-b0da-5cf03c97af44 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:52.492615 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfc7105-430a-4c02-a585-ba57500e4980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:52.493290 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfc7105-430a-4c02-a585-ba57500e4980 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:52.493618 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfc7105-430a-4c02-a585-ba57500e4980 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:52.493896 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfc7105-430a-4c02-a585-ba57500e4980 None admin] PATH_INFO: `/v3/users` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:52.494734 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfc7105-430a-4c02-a585-ba57500e4980 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:58:52.495043 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfc7105-430a-4c02-a585-ba57500e4980 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:52.496485 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfc7105-430a-4c02-a585-ba57500e4980 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:52.533465 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 255/497] 10.222.0.23 () {56 vars in 1133 bytes} [Sat Jun 18 04:58:52 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:52.534777 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a5d700-c005-4db0-9568-cb2b5e81f40b None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:52.535067 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a5d700-c005-4db0-9568-cb2b5e81f40b None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:52.535338 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a5d700-c005-4db0-9568-cb2b5e81f40b None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:52.540829 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 256/498] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 04:58:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 04:58:52.540829 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a5d700-c005-4db0-9568-cb2b5e81f40b None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:52.592675 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:52.605150 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3732185-ece0-447e-b0da-5cf03c97af44 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:52.613786 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:52.642635 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9c72fb-0667-4811-9fee-2fafdac58959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:52.643623 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9c72fb-0667-4811-9fee-2fafdac58959 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:52.644217 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9c72fb-0667-4811-9fee-2fafdac58959 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:52.644515 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9c72fb-0667-4811-9fee-2fafdac58959 None admin] PATH_INFO: `/v3/domains` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:52.645439 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9c72fb-0667-4811-9fee-2fafdac58959 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:58:52.645760 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9c72fb-0667-4811-9fee-2fafdac58959 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:52.647848 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9c72fb-0667-4811-9fee-2fafdac58959 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:52.649717 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a5d700-c005-4db0-9568-cb2b5e81f40b None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:52.666915 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 243/499] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Jun 18 04:58:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:52.666915 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9c72fb-0667-4811-9fee-2fafdac58959 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:52.684512 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224d8cff-1fb3-4efa-bffb-e33d8f6c33e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:52.685298 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224d8cff-1fb3-4efa-bffb-e33d8f6c33e8 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:52.685611 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224d8cff-1fb3-4efa-bffb-e33d8f6c33e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:52.685882 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224d8cff-1fb3-4efa-bffb-e33d8f6c33e8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:52.696871 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224d8cff-1fb3-4efa-bffb-e33d8f6c33e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:52.698363 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224d8cff-1fb3-4efa-bffb-e33d8f6c33e8 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:52.716313 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-224d8cff-1fb3-4efa-bffb-e33d8f6c33e8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 04:58:52.716313 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:58:52.716313 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:58:52.716313 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:58:52.716313 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:58:52.716313 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:58:52.716313 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:58:52.716313 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:58:52.716313 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:58:52.716313 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:58:52.716313 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:58:52.716313 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:58:52.716313 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 18 04:58:52.716313 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 18 04:58:52.716313 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 18 04:58:52.717243 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 18 04:58:52.717243 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:58:52.717243 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:58:52.717243 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:58:52.717243 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:58:52.717243 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:58:52.717243 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:58:52.717243 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:58:52.717243 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jun 18 04:58:52.717243 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:58:52.717243 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:58:52.717243 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:58:52.717243 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:58:52.717243 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:58:52.717243 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:58:52.717243 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:58:52.717243 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:58:52.718140 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 18 04:58:52.718140 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 18 04:58:52.718140 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 04:58:52.718140 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 04:58:52.722353 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 257/500] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Jun 18 04:58:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:58:52.723353 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e9445d-7a22-41d6-ad41-d85475e1a82b None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:52.723657 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e9445d-7a22-41d6-ad41-d85475e1a82b None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:52.724024 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e9445d-7a22-41d6-ad41-d85475e1a82b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:52.741590 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-e7e9445d-7a22-41d6-ad41-d85475e1a82b None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 04:58:52.742805 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-e7e9445d-7a22-41d6-ad41-d85475e1a82b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 04:58:52.781998 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9c72fb-0667-4811-9fee-2fafdac58959 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:52.812147 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 258/501] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 04:58:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 18 04:58:52.812983 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e9445d-7a22-41d6-ad41-d85475e1a82b None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:52.817306 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbd5cba-8b78-4a22-abe6-dfbaed9ba898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:52.817973 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbd5cba-8b78-4a22-abe6-dfbaed9ba898 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:52.818973 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbd5cba-8b78-4a22-abe6-dfbaed9ba898 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:52.819255 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbd5cba-8b78-4a22-abe6-dfbaed9ba898 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:52.840981 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:52.870839 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:52.891513 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbd5cba-8b78-4a22-abe6-dfbaed9ba898 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:52.892974 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbd5cba-8b78-4a22-abe6-dfbaed9ba898 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:52.920701 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e9445d-7a22-41d6-ad41-d85475e1a82b None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:52.954600 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034c93a4-5632-4a96-9126-bd53ce41ee9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:52.955379 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034c93a4-5632-4a96-9126-bd53ce41ee9d None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:52.955658 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034c93a4-5632-4a96-9126-bd53ce41ee9d None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:52.955934 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034c93a4-5632-4a96-9126-bd53ce41ee9d None admin] PATH_INFO: `/v3/users/glance` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:52.962830 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034c93a4-5632-4a96-9126-bd53ce41ee9d None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:52.964273 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034c93a4-5632-4a96-9126-bd53ce41ee9d None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:52.980334 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-034c93a4-5632-4a96-9126-bd53ce41ee9d None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 18 04:58:52.980334 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:58:52.980334 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:58:52.980334 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:58:52.980334 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:58:52.980334 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:58:52.980334 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:58:52.980334 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:58:52.980334 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:58:52.980334 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:58:52.980334 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:58:52.980334 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:58:52.980334 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 18 04:58:52.980334 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 18 04:58:52.980334 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 18 04:58:52.980334 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 18 04:58:52.981227 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:58:52.981227 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:58:52.981227 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 18 04:58:52.981227 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 18 04:58:52.981227 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 18 04:58:52.981227 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 18 04:58:52.981227 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:58:52.981227 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:58:52.981227 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:58:52.981227 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:58:52.981227 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:58:52.981227 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jun 18 04:58:52.981227 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:58:52.981227 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:58:52.981227 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:58:52.981227 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:58:52.981227 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:58:52.981227 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:58:52.982176 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:58:52.982176 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:58:52.982176 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 18 04:58:52.982176 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 18 04:58:52.982176 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 18 04:58:52.982176 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 18 04:58:52.982176 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 18 04:58:52.982176 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 18 04:58:52.982176 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 18 04:58:52.982176 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 04:58:52.982176 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 259/502] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Jun 18 04:58:52 2022] GET /identity/v3/users/glance => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:58:52.982727 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034c93a4-5632-4a96-9126-bd53ce41ee9d None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:52.998439 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 244/503] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Jun 18 04:58:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 18 04:58:52.999894 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5328a848-e403-4e36-9d62-ce892141a743 None None] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:53.000192 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5328a848-e403-4e36-9d62-ce892141a743 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:53.000472 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5328a848-e403-4e36-9d62-ce892141a743 None None] PATH_INFO: `/` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:53.001892 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 245/504] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 04:58:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 04:58:53.002516 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5328a848-e403-4e36-9d62-ce892141a743 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:53.053674 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:53.085598 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034c93a4-5632-4a96-9126-bd53ce41ee9d None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:53.096572 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:53.105171 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5328a848-e403-4e36-9d62-ce892141a743 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:53.119836 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad93b0c8-2eed-4095-87a1-b5c9ef5f8f76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:53.120490 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad93b0c8-2eed-4095-87a1-b5c9ef5f8f76 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:53.120776 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad93b0c8-2eed-4095-87a1-b5c9ef5f8f76 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:53.121101 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad93b0c8-2eed-4095-87a1-b5c9ef5f8f76 None admin] PATH_INFO: `/v3/domains` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:53.121808 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad93b0c8-2eed-4095-87a1-b5c9ef5f8f76 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:58:53.122122 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad93b0c8-2eed-4095-87a1-b5c9ef5f8f76 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:53.123580 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad93b0c8-2eed-4095-87a1-b5c9ef5f8f76 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:53.135801 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 260/505] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Jun 18 04:58:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:53.136581 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad93b0c8-2eed-4095-87a1-b5c9ef5f8f76 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:53.139279 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a270e4f1-203f-446d-94a4-ff74fcd78389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:53.140264 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a270e4f1-203f-446d-94a4-ff74fcd78389 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:53.140433 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a270e4f1-203f-446d-94a4-ff74fcd78389 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:53.140711 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a270e4f1-203f-446d-94a4-ff74fcd78389 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:53.151031 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a270e4f1-203f-446d-94a4-ff74fcd78389 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 18 04:58:53.158634 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application [None req-a270e4f1-203f-446d-94a4-ff74fcd78389 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 18 04:58:53.158634 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:58:53.158634 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:58:53.158634 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:58:53.158634 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:58:53.158634 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:58:53.158634 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:58:53.158634 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:58:53.158634 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:58:53.158634 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:58:53.158634 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:58:53.158634 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:58:53.158634 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 18 04:58:53.158634 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 18 04:58:53.158634 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 18 04:58:53.158634 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 18 04:58:53.159505 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 18 04:58:53.159505 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 18 04:58:53.159505 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 18 04:58:53.159505 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 18 04:58:53.159505 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:58:53.159505 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:58:53.159505 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:58:53.159505 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:58:53.159505 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:58:53.159505 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:58:53.159505 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:58:53.159505 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application with Lock( Jun 18 04:58:53.159505 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:58:53.159505 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:58:53.159505 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:58:53.159505 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:58:53.160295 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:58:53.160295 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:58:53.160295 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:58:53.160295 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:58:53.160295 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 18 04:58:53.160295 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 18 04:58:53.160295 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 18 04:58:53.160295 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 18 04:58:53.160295 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 18 04:58:53.160295 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 18 04:58:53.160295 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 18 04:58:53.160295 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 04:58:53.160295 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 246/506] 10.222.0.23 () {56 vars in 1102 bytes} [Sat Jun 18 04:58:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:58:53.161018 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e686a3da-b2de-4a72-a26c-d17ba7c5a86f None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:53.161305 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e686a3da-b2de-4a72-a26c-d17ba7c5a86f None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:53.161611 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e686a3da-b2de-4a72-a26c-d17ba7c5a86f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:53.179727 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-e686a3da-b2de-4a72-a26c-d17ba7c5a86f None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 04:58:53.180922 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-e686a3da-b2de-4a72-a26c-d17ba7c5a86f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 04:58:53.239749 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad93b0c8-2eed-4095-87a1-b5c9ef5f8f76 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:53.248487 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 247/507] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 04:58:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 18 04:58:53.249325 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e686a3da-b2de-4a72-a26c-d17ba7c5a86f None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:53.282289 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a7400d-3c25-4110-9eba-706ebe758304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:53.282941 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a7400d-3c25-4110-9eba-706ebe758304 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:53.283221 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a7400d-3c25-4110-9eba-706ebe758304 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:53.283491 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a7400d-3c25-4110-9eba-706ebe758304 None admin] PATH_INFO: `/v3/users` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:53.284313 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a7400d-3c25-4110-9eba-706ebe758304 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:58:53.284617 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a7400d-3c25-4110-9eba-706ebe758304 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:53.286081 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a7400d-3c25-4110-9eba-706ebe758304 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:53.293797 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:53.304467 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 261/508] 10.222.0.23 () {56 vars in 1137 bytes} [Sat Jun 18 04:58:53 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:53.305323 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a7400d-3c25-4110-9eba-706ebe758304 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:53.332969 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:53.353022 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e686a3da-b2de-4a72-a26c-d17ba7c5a86f None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:53.388776 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d83f35f-6f6c-4d53-999f-e42212b7a826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:53.389519 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d83f35f-6f6c-4d53-999f-e42212b7a826 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:53.389802 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d83f35f-6f6c-4d53-999f-e42212b7a826 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:53.390172 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d83f35f-6f6c-4d53-999f-e42212b7a826 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:53.400072 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d83f35f-6f6c-4d53-999f-e42212b7a826 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:53.401559 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d83f35f-6f6c-4d53-999f-e42212b7a826 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:53.407844 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a7400d-3c25-4110-9eba-706ebe758304 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:53.419020 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application [None req-6d83f35f-6f6c-4d53-999f-e42212b7a826 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 04:58:53.419020 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:58:53.419020 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:58:53.419020 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:58:53.419020 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:58:53.419020 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:58:53.419020 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:58:53.419020 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:58:53.419020 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:58:53.419020 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:58:53.419020 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:58:53.419020 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:58:53.419020 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 18 04:58:53.419020 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 18 04:58:53.419020 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 18 04:58:53.419852 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 18 04:58:53.419852 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:58:53.419852 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:58:53.419852 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:58:53.419852 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:58:53.419852 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:58:53.419852 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:58:53.419852 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:58:53.419852 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application with Lock( Jun 18 04:58:53.419852 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:58:53.419852 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:58:53.419852 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:58:53.419852 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:58:53.419852 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:58:53.419852 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:58:53.419852 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:58:53.419852 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:58:53.420761 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 18 04:58:53.420761 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 18 04:58:53.420761 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 04:58:53.420761 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 04:58:53.424667 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 248/509] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Jun 18 04:58:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:58:53.425438 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d83f35f-6f6c-4d53-999f-e42212b7a826 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:53.445960 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788b44ad-1195-48e2-a052-58360435dc97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:53.446753 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788b44ad-1195-48e2-a052-58360435dc97 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:53.447041 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788b44ad-1195-48e2-a052-58360435dc97 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:53.447322 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788b44ad-1195-48e2-a052-58360435dc97 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:53.454240 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788b44ad-1195-48e2-a052-58360435dc97 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:53.455684 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788b44ad-1195-48e2-a052-58360435dc97 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:53.469784 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-788b44ad-1195-48e2-a052-58360435dc97 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 18 04:58:53.469784 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:58:53.469784 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:58:53.469784 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:58:53.469784 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:58:53.469784 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:58:53.469784 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:58:53.469784 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:58:53.469784 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:58:53.469784 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:58:53.469784 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:58:53.469784 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:58:53.469784 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 18 04:58:53.469784 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 18 04:58:53.469784 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 18 04:58:53.469784 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 18 04:58:53.470839 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:58:53.470839 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:58:53.470839 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 18 04:58:53.470839 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 18 04:58:53.470839 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 18 04:58:53.470839 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 18 04:58:53.470839 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:58:53.470839 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:58:53.470839 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:58:53.470839 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:58:53.470839 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:58:53.470839 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jun 18 04:58:53.470839 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:58:53.470839 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:58:53.470839 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:58:53.470839 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:58:53.470839 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:58:53.470839 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:58:53.471855 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:58:53.471855 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:58:53.471855 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 18 04:58:53.471855 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 18 04:58:53.471855 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 18 04:58:53.471855 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 18 04:58:53.471855 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 18 04:58:53.471855 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 18 04:58:53.471855 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 18 04:58:53.471855 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 04:58:53.471855 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 262/510] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Jun 18 04:58:53 2022] GET /identity/v3/users/cinder => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:58:53.471855 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788b44ad-1195-48e2-a052-58360435dc97 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:53.519053 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:53.528992 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d83f35f-6f6c-4d53-999f-e42212b7a826 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:53.559799 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:53.563687 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e294485-8d12-44df-ae12-cda8af14b2e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:53.564348 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e294485-8d12-44df-ae12-cda8af14b2e2 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:53.564628 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e294485-8d12-44df-ae12-cda8af14b2e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:53.564901 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e294485-8d12-44df-ae12-cda8af14b2e2 None admin] PATH_INFO: `/v3/roles` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:53.565671 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e294485-8d12-44df-ae12-cda8af14b2e2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:58:53.565986 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e294485-8d12-44df-ae12-cda8af14b2e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:53.567346 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e294485-8d12-44df-ae12-cda8af14b2e2 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:53.572301 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788b44ad-1195-48e2-a052-58360435dc97 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:53.575524 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 249/511] 10.222.0.23 () {56 vars in 1103 bytes} [Sat Jun 18 04:58:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:53.576304 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e294485-8d12-44df-ae12-cda8af14b2e2 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:53.609435 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e44d3b2-770d-4d0d-9cd0-f476ff148058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:53.610231 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e44d3b2-770d-4d0d-9cd0-f476ff148058 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:53.610513 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e44d3b2-770d-4d0d-9cd0-f476ff148058 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:53.610785 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e44d3b2-770d-4d0d-9cd0-f476ff148058 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:53.621147 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e44d3b2-770d-4d0d-9cd0-f476ff148058 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:53.622636 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e44d3b2-770d-4d0d-9cd0-f476ff148058 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:53.640637 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-4e44d3b2-770d-4d0d-9cd0-f476ff148058 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 04:58:53.640637 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:58:53.640637 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:58:53.640637 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:58:53.640637 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:58:53.640637 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:58:53.640637 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:58:53.640637 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:58:53.640637 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:58:53.640637 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:58:53.640637 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:58:53.640637 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:58:53.640637 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 18 04:58:53.640637 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 18 04:58:53.640637 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 18 04:58:53.641513 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 18 04:58:53.641513 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:58:53.641513 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:58:53.641513 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:58:53.641513 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:58:53.641513 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:58:53.641513 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:58:53.641513 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:58:53.641513 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jun 18 04:58:53.641513 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:58:53.641513 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:58:53.641513 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:58:53.641513 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:58:53.641513 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:58:53.641513 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:58:53.641513 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:58:53.641513 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:58:53.642340 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 18 04:58:53.642340 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 18 04:58:53.642340 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 04:58:53.642340 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 04:58:53.642340 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 263/512] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Jun 18 04:58:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:58:53.642605 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e44d3b2-770d-4d0d-9cd0-f476ff148058 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:53.681663 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e294485-8d12-44df-ae12-cda8af14b2e2 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:53.715514 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a830c0-85f5-46b4-aad8-88bc7cb66b52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:53.716154 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a830c0-85f5-46b4-aad8-88bc7cb66b52 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:53.716432 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a830c0-85f5-46b4-aad8-88bc7cb66b52 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:53.716710 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a830c0-85f5-46b4-aad8-88bc7cb66b52 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:53.738284 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:53.744868 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e44d3b2-770d-4d0d-9cd0-f476ff148058 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:53.778812 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4541a0-c52c-4592-8ae0-73bdf591e45a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:53.779467 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4541a0-c52c-4592-8ae0-73bdf591e45a None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:53.779742 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4541a0-c52c-4592-8ae0-73bdf591e45a None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:53.780030 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4541a0-c52c-4592-8ae0-73bdf591e45a None admin] PATH_INFO: `/v3/domains` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:53.780745 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4541a0-c52c-4592-8ae0-73bdf591e45a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:58:53.781053 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4541a0-c52c-4592-8ae0-73bdf591e45a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:53.782458 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4541a0-c52c-4592-8ae0-73bdf591e45a None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:53.786229 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a830c0-85f5-46b4-aad8-88bc7cb66b52 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:53.787768 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a830c0-85f5-46b4-aad8-88bc7cb66b52 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:53.791449 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:53.794387 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 264/513] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Jun 18 04:58:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:53.795169 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4541a0-c52c-4592-8ae0-73bdf591e45a None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:53.889609 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 250/514] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Jun 18 04:58:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 18 04:58:53.890442 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a830c0-85f5-46b4-aad8-88bc7cb66b52 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:53.894219 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4541a0-c52c-4592-8ae0-73bdf591e45a None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:53.928928 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7133ee72-2a4f-4957-9bb0-8b46a6aafb89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:53.929612 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7133ee72-2a4f-4957-9bb0-8b46a6aafb89 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:53.929893 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7133ee72-2a4f-4957-9bb0-8b46a6aafb89 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:53.930164 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7133ee72-2a4f-4957-9bb0-8b46a6aafb89 None admin] PATH_INFO: `/v3/users` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:53.930975 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7133ee72-2a4f-4957-9bb0-8b46a6aafb89 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:58:53.931295 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7133ee72-2a4f-4957-9bb0-8b46a6aafb89 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:53.932717 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7133ee72-2a4f-4957-9bb0-8b46a6aafb89 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:53.943594 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:53.949572 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 265/515] 10.222.0.23 () {56 vars in 1137 bytes} [Sat Jun 18 04:58:53 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:53.950356 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7133ee72-2a4f-4957-9bb0-8b46a6aafb89 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:53.991176 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a830c0-85f5-46b4-aad8-88bc7cb66b52 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:54.008802 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:54.026037 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647a8334-38d5-481e-9901-61541d8bf6cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:54.026808 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647a8334-38d5-481e-9901-61541d8bf6cb None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:54.027090 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647a8334-38d5-481e-9901-61541d8bf6cb None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:54.027363 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647a8334-38d5-481e-9901-61541d8bf6cb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:54.038327 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647a8334-38d5-481e-9901-61541d8bf6cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:54.039772 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647a8334-38d5-481e-9901-61541d8bf6cb None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:54.051225 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7133ee72-2a4f-4957-9bb0-8b46a6aafb89 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:54.056887 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application [None req-647a8334-38d5-481e-9901-61541d8bf6cb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 04:58:54.056887 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:58:54.056887 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:58:54.056887 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:58:54.056887 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:58:54.056887 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:58:54.056887 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:58:54.056887 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:58:54.056887 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:58:54.056887 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:58:54.056887 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:58:54.056887 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:58:54.056887 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 18 04:58:54.056887 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 18 04:58:54.056887 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 18 04:58:54.057893 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 18 04:58:54.057893 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:58:54.057893 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:58:54.057893 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:58:54.057893 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:58:54.057893 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:58:54.057893 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:58:54.057893 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:58:54.057893 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application with Lock( Jun 18 04:58:54.057893 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:58:54.057893 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:58:54.057893 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:58:54.057893 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:58:54.057893 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:58:54.057893 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:58:54.057893 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:58:54.057893 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:58:54.058905 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 18 04:58:54.058905 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 18 04:58:54.058905 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 04:58:54.058905 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 04:58:54.058905 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 251/516] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Jun 18 04:58:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:58:54.059226 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647a8334-38d5-481e-9901-61541d8bf6cb None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:54.086404 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2ee2d5-9ce3-42db-a459-2a70b7764208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:54.087063 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2ee2d5-9ce3-42db-a459-2a70b7764208 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:54.087348 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2ee2d5-9ce3-42db-a459-2a70b7764208 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:54.087623 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2ee2d5-9ce3-42db-a459-2a70b7764208 None admin] PATH_INFO: `/v3/domains` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:54.088339 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2ee2d5-9ce3-42db-a459-2a70b7764208 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:58:54.088651 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2ee2d5-9ce3-42db-a459-2a70b7764208 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:54.090054 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2ee2d5-9ce3-42db-a459-2a70b7764208 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:54.101536 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 266/517] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Jun 18 04:58:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:54.102469 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2ee2d5-9ce3-42db-a459-2a70b7764208 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:54.159172 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647a8334-38d5-481e-9901-61541d8bf6cb None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:54.161363 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:54.192647 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73ca667-fc5a-47e1-89fe-7814c61ba7a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:54.194128 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73ca667-fc5a-47e1-89fe-7814c61ba7a8 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:54.194418 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73ca667-fc5a-47e1-89fe-7814c61ba7a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:54.194698 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73ca667-fc5a-47e1-89fe-7814c61ba7a8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:54.201547 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73ca667-fc5a-47e1-89fe-7814c61ba7a8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:54.202103 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2ee2d5-9ce3-42db-a459-2a70b7764208 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:54.203274 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73ca667-fc5a-47e1-89fe-7814c61ba7a8 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:54.217828 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application [None req-c73ca667-fc5a-47e1-89fe-7814c61ba7a8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 18 04:58:54.217828 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:58:54.217828 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:58:54.217828 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:58:54.217828 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:58:54.217828 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:58:54.217828 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:58:54.217828 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:58:54.217828 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:58:54.217828 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:58:54.217828 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:58:54.217828 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:58:54.217828 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 18 04:58:54.217828 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 18 04:58:54.217828 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 18 04:58:54.218751 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 18 04:58:54.218751 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:58:54.218751 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:58:54.218751 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:58:54.218751 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:58:54.218751 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:58:54.218751 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:58:54.218751 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:58:54.218751 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application with Lock( Jun 18 04:58:54.218751 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:58:54.218751 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:58:54.218751 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:58:54.218751 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:58:54.218751 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:58:54.218751 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:58:54.218751 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:58:54.218751 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:58:54.219617 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 18 04:58:54.219617 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 18 04:58:54.219617 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 18 04:58:54.219617 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 18 04:58:54.219617 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 18 04:58:54.219617 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 18 04:58:54.219617 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 18 04:58:54.219617 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 04:58:54.219617 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 252/518] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Jun 18 04:58:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:58:54.220084 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73ca667-fc5a-47e1-89fe-7814c61ba7a8 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:54.221288 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:54.237836 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4a92ea-cede-4de2-a4d2-a273ce189042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:54.238623 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4a92ea-cede-4de2-a4d2-a273ce189042 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:54.238907 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4a92ea-cede-4de2-a4d2-a273ce189042 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:54.239182 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4a92ea-cede-4de2-a4d2-a273ce189042 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:54.250565 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4a92ea-cede-4de2-a4d2-a273ce189042 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:54.252159 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4a92ea-cede-4de2-a4d2-a273ce189042 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:54.269769 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-5e4a92ea-cede-4de2-a4d2-a273ce189042 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 04:58:54.269769 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:58:54.269769 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:58:54.269769 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:58:54.269769 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:58:54.269769 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:58:54.269769 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:58:54.269769 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:58:54.269769 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:58:54.269769 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:58:54.269769 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:58:54.269769 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:58:54.269769 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 18 04:58:54.269769 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 18 04:58:54.269769 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 18 04:58:54.270811 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 18 04:58:54.270811 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:58:54.270811 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:58:54.270811 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:58:54.270811 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:58:54.270811 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:58:54.270811 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:58:54.270811 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:58:54.270811 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jun 18 04:58:54.270811 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:58:54.270811 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:58:54.270811 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:58:54.270811 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:58:54.270811 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:58:54.270811 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:58:54.270811 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:58:54.270811 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:58:54.271829 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 18 04:58:54.271829 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 18 04:58:54.271829 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 04:58:54.271829 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 04:58:54.271829 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 267/519] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Jun 18 04:58:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:58:54.272518 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4a92ea-cede-4de2-a4d2-a273ce189042 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:54.324222 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73ca667-fc5a-47e1-89fe-7814c61ba7a8 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:54.357932 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd876647-d7d7-4abb-b865-67b2b08675ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:54.358567 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd876647-d7d7-4abb-b865-67b2b08675ff None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:54.358851 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd876647-d7d7-4abb-b865-67b2b08675ff None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:54.359127 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd876647-d7d7-4abb-b865-67b2b08675ff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:54.372122 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4a92ea-cede-4de2-a4d2-a273ce189042 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:54.391391 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:54.408310 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5728bb-404d-4061-8aee-5dad495b64bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:54.408973 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5728bb-404d-4061-8aee-5dad495b64bb None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:54.409308 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5728bb-404d-4061-8aee-5dad495b64bb None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:54.409590 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5728bb-404d-4061-8aee-5dad495b64bb None admin] PATH_INFO: `/v3/domains` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:54.410308 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5728bb-404d-4061-8aee-5dad495b64bb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:58:54.410622 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5728bb-404d-4061-8aee-5dad495b64bb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:54.412556 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5728bb-404d-4061-8aee-5dad495b64bb None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:54.424238 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 268/520] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Jun 18 04:58:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:54.425007 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5728bb-404d-4061-8aee-5dad495b64bb None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:54.432562 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd876647-d7d7-4abb-b865-67b2b08675ff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:54.434071 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd876647-d7d7-4abb-b865-67b2b08675ff None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:54.454363 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:54.522713 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5728bb-404d-4061-8aee-5dad495b64bb None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:54.532874 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 253/521] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Jun 18 04:58:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 18 04:58:54.533806 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd876647-d7d7-4abb-b865-67b2b08675ff None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:54.557271 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4027582e-8937-430f-9768-b2e33e8a9314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:54.557958 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4027582e-8937-430f-9768-b2e33e8a9314 None admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:54.558234 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4027582e-8937-430f-9768-b2e33e8a9314 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:54.558505 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4027582e-8937-430f-9768-b2e33e8a9314 None admin] PATH_INFO: `/v3/users` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:54.559195 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4027582e-8937-430f-9768-b2e33e8a9314 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:54.560633 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4027582e-8937-430f-9768-b2e33e8a9314 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:54.594902 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:54.631762 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd876647-d7d7-4abb-b865-67b2b08675ff None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:54.636266 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 269/522] 10.222.0.23 () {60 vars in 1132 bytes} [Sat Jun 18 04:58:54 2022] POST /identity/v3/users => generated 247 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 04:58:54.637090 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4027582e-8937-430f-9768-b2e33e8a9314 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:54.678582 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b825dee1-cbea-4516-b7a3-84a2b9e6e104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:54.679415 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b825dee1-cbea-4516-b7a3-84a2b9e6e104 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:54.679793 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b825dee1-cbea-4516-b7a3-84a2b9e6e104 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:54.680164 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b825dee1-cbea-4516-b7a3-84a2b9e6e104 None admin] PATH_INFO: `/v3/projects` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:54.681190 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b825dee1-cbea-4516-b7a3-84a2b9e6e104 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:58:54.681641 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b825dee1-cbea-4516-b7a3-84a2b9e6e104 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:54.683233 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b825dee1-cbea-4516-b7a3-84a2b9e6e104 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:54.704628 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 254/523] 10.222.0.23 () {56 vars in 1148 bytes} [Sat Jun 18 04:58:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:54.705744 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b825dee1-cbea-4516-b7a3-84a2b9e6e104 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:54.742333 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:54.758225 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4027582e-8937-430f-9768-b2e33e8a9314 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:54.797040 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f09729-971e-42cf-afc6-9ff137868dfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:54.797771 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f09729-971e-42cf-afc6-9ff137868dfd None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:54.798207 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f09729-971e-42cf-afc6-9ff137868dfd None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:54.798485 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f09729-971e-42cf-afc6-9ff137868dfd None admin] PATH_INFO: `/v3/domains` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:54.799221 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f09729-971e-42cf-afc6-9ff137868dfd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:58:54.799518 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f09729-971e-42cf-afc6-9ff137868dfd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:54.800934 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f09729-971e-42cf-afc6-9ff137868dfd None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:54.814496 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 270/524] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Jun 18 04:58:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:54.815872 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f09729-971e-42cf-afc6-9ff137868dfd None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:54.832208 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b825dee1-cbea-4516-b7a3-84a2b9e6e104 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:54.859217 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:54.869883 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e312274f-0ffb-4bd3-82a0-6647d0b6bbad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:54.870679 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e312274f-0ffb-4bd3-82a0-6647d0b6bbad None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:54.870966 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e312274f-0ffb-4bd3-82a0-6647d0b6bbad None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:54.871259 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e312274f-0ffb-4bd3-82a0-6647d0b6bbad None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:54.878486 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e312274f-0ffb-4bd3-82a0-6647d0b6bbad None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:54.879925 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e312274f-0ffb-4bd3-82a0-6647d0b6bbad None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:54.894614 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application [None req-e312274f-0ffb-4bd3-82a0-6647d0b6bbad None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 18 04:58:54.894614 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:58:54.894614 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:58:54.894614 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:58:54.894614 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:58:54.894614 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:58:54.894614 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:58:54.894614 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:58:54.894614 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:58:54.894614 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:58:54.894614 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:58:54.894614 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:58:54.894614 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 18 04:58:54.894614 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 18 04:58:54.894614 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 18 04:58:54.894614 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 18 04:58:54.895663 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:58:54.895663 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:58:54.895663 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 18 04:58:54.895663 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 18 04:58:54.895663 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 18 04:58:54.895663 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 18 04:58:54.895663 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:58:54.895663 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:58:54.895663 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:58:54.895663 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:58:54.895663 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:58:54.895663 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application with Lock( Jun 18 04:58:54.895663 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:58:54.895663 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:58:54.895663 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:58:54.895663 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:58:54.895663 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:58:54.895663 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:58:54.897578 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:58:54.897578 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:58:54.897578 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 18 04:58:54.897578 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 18 04:58:54.897578 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 18 04:58:54.897578 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 18 04:58:54.897578 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 18 04:58:54.897578 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 18 04:58:54.897578 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 18 04:58:54.897578 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 04:58:54.904235 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 255/525] 10.222.0.23 () {56 vars in 1102 bytes} [Sat Jun 18 04:58:54 2022] GET /identity/v3/users/neutron => generated 85 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:58:54.905163 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e312274f-0ffb-4bd3-82a0-6647d0b6bbad None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:54.934298 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f09729-971e-42cf-afc6-9ff137868dfd None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:54.970596 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87db1c9-6357-4029-94eb-3ffd25acba81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:54.971391 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87db1c9-6357-4029-94eb-3ffd25acba81 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:54.971675 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87db1c9-6357-4029-94eb-3ffd25acba81 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:54.971955 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87db1c9-6357-4029-94eb-3ffd25acba81 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:54.982553 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87db1c9-6357-4029-94eb-3ffd25acba81 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:54.984023 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87db1c9-6357-4029-94eb-3ffd25acba81 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:54.991861 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:55.003107 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-b87db1c9-6357-4029-94eb-3ffd25acba81 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 04:58:55.003107 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:58:55.003107 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:58:55.003107 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:58:55.003107 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:58:55.003107 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:58:55.003107 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:58:55.003107 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:58:55.003107 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:58:55.003107 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:58:55.003107 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:58:55.003107 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:58:55.003107 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 18 04:58:55.003107 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 18 04:58:55.003107 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 18 04:58:55.004242 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 18 04:58:55.004242 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:58:55.004242 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:58:55.004242 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:58:55.004242 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:58:55.004242 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:58:55.004242 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:58:55.004242 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:58:55.004242 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jun 18 04:58:55.004242 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:58:55.004242 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:58:55.004242 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:58:55.004242 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:58:55.004242 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:58:55.004242 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:58:55.004242 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:58:55.004242 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:58:55.005129 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 18 04:58:55.005129 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 18 04:58:55.005129 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 04:58:55.005129 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 04:58:55.005129 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 271/526] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Jun 18 04:58:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:58:55.005419 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87db1c9-6357-4029-94eb-3ffd25acba81 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:55.021986 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e312274f-0ffb-4bd3-82a0-6647d0b6bbad None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:55.057774 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363df901-6c61-4523-9bea-48435886a27c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:55.058421 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363df901-6c61-4523-9bea-48435886a27c None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:55.058702 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363df901-6c61-4523-9bea-48435886a27c None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:55.059034 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363df901-6c61-4523-9bea-48435886a27c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:55.059867 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363df901-6c61-4523-9bea-48435886a27c None admin] RBAC: Adding query filter params (role.id=5ec7b68b793d4a54abd30460961346b9, scope.project.id=c95f8f3084c5472cb8364b7c57207394, user.id=15e7f45ee0744c1d9602187eac8e8b80) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:58:55.060225 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363df901-6c61-4523-9bea-48435886a27c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:55.061687 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363df901-6c61-4523-9bea-48435886a27c None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:55.067250 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 256/527] 10.222.0.23 () {56 vars in 1374 bytes} [Sat Jun 18 04:58:54 2022] GET /identity/v3/role_assignments?user.id=15e7f45ee0744c1d9602187eac8e8b80&scope.project.id=c95f8f3084c5472cb8364b7c57207394&role.id=5ec7b68b793d4a54abd30460961346b9 => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:55.068016 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363df901-6c61-4523-9bea-48435886a27c None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:55.130072 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:55.134401 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87db1c9-6357-4029-94eb-3ffd25acba81 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:55.187757 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9407a6-08d9-4a42-9dcd-243a72764aab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:55.189065 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9407a6-08d9-4a42-9dcd-243a72764aab None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:55.189776 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9407a6-08d9-4a42-9dcd-243a72764aab None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:55.190229 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9407a6-08d9-4a42-9dcd-243a72764aab None admin] PATH_INFO: `/v3/projects/service` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:55.194262 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363df901-6c61-4523-9bea-48435886a27c None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:55.200569 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9407a6-08d9-4a42-9dcd-243a72764aab None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:55.202298 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9407a6-08d9-4a42-9dcd-243a72764aab None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:55.222840 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-7a9407a6-08d9-4a42-9dcd-243a72764aab None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 18 04:58:55.222840 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:58:55.222840 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:58:55.222840 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:58:55.222840 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:58:55.222840 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:58:55.222840 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:58:55.222840 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:58:55.222840 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:58:55.222840 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:58:55.222840 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:58:55.222840 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:58:55.222840 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 18 04:58:55.222840 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 18 04:58:55.222840 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 18 04:58:55.223971 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 18 04:58:55.223971 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:58:55.223971 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:58:55.223971 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:58:55.223971 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:58:55.223971 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:58:55.223971 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:58:55.223971 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:58:55.223971 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jun 18 04:58:55.223971 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:58:55.223971 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:58:55.223971 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:58:55.223971 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:58:55.223971 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:58:55.223971 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:58:55.223971 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:58:55.223971 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:58:55.226246 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 18 04:58:55.226246 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 18 04:58:55.226246 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 18 04:58:55.226246 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 18 04:58:55.226246 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 18 04:58:55.226246 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 18 04:58:55.226246 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 18 04:58:55.226246 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 04:58:55.229541 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 272/528] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Jun 18 04:58:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:58:55.230601 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9407a6-08d9-4a42-9dcd-243a72764aab None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:55.243704 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e1bb8e-a7a3-42ad-b609-990e79213c0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:55.244744 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e1bb8e-a7a3-42ad-b609-990e79213c0a None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:55.245201 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e1bb8e-a7a3-42ad-b609-990e79213c0a None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:55.245787 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e1bb8e-a7a3-42ad-b609-990e79213c0a None admin] PATH_INFO: `/v3/users` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:55.246820 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e1bb8e-a7a3-42ad-b609-990e79213c0a None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:58:55.247303 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e1bb8e-a7a3-42ad-b609-990e79213c0a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:55.249147 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e1bb8e-a7a3-42ad-b609-990e79213c0a None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:55.295759 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 257/529] 10.222.0.23 () {56 vars in 1139 bytes} [Sat Jun 18 04:58:55 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:55.296526 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e1bb8e-a7a3-42ad-b609-990e79213c0a None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:55.304548 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:55.357317 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9407a6-08d9-4a42-9dcd-243a72764aab None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:55.400787 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f95d0db-11bc-426a-97e0-ea983a016290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:55.401490 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f95d0db-11bc-426a-97e0-ea983a016290 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:55.401776 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f95d0db-11bc-426a-97e0-ea983a016290 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:55.402231 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f95d0db-11bc-426a-97e0-ea983a016290 None admin] PATH_INFO: `/v3/domains` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:55.402958 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f95d0db-11bc-426a-97e0-ea983a016290 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:58:55.403268 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f95d0db-11bc-426a-97e0-ea983a016290 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:55.404640 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f95d0db-11bc-426a-97e0-ea983a016290 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:55.419161 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 273/530] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Jun 18 04:58:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:55.420081 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f95d0db-11bc-426a-97e0-ea983a016290 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:55.438063 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e1bb8e-a7a3-42ad-b609-990e79213c0a None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:55.446689 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:55.477837 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9424d363-baca-438f-ae8b-40b4107ed6cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:55.478666 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9424d363-baca-438f-ae8b-40b4107ed6cf None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:55.479053 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9424d363-baca-438f-ae8b-40b4107ed6cf None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:55.479480 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9424d363-baca-438f-ae8b-40b4107ed6cf None admin] PATH_INFO: `/v3/projects` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:55.480328 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9424d363-baca-438f-ae8b-40b4107ed6cf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:58:55.480793 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9424d363-baca-438f-ae8b-40b4107ed6cf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:55.483265 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9424d363-baca-438f-ae8b-40b4107ed6cf None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:55.500615 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 258/531] 10.222.0.23 () {56 vars in 1148 bytes} [Sat Jun 18 04:58:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:55.500615 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9424d363-baca-438f-ae8b-40b4107ed6cf None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:55.535328 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f95d0db-11bc-426a-97e0-ea983a016290 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:55.554865 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:55.574389 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879c17a7-5421-45c6-8d52-d89e29178151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:55.575659 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879c17a7-5421-45c6-8d52-d89e29178151 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:55.576138 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879c17a7-5421-45c6-8d52-d89e29178151 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:55.577234 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879c17a7-5421-45c6-8d52-d89e29178151 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:55.620181 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9424d363-baca-438f-ae8b-40b4107ed6cf None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:55.667563 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9133758-b3ef-45f7-8870-dbae63c95c30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:55.668365 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9133758-b3ef-45f7-8870-dbae63c95c30 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:55.668652 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9133758-b3ef-45f7-8870-dbae63c95c30 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:55.668951 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9133758-b3ef-45f7-8870-dbae63c95c30 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:55.669886 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879c17a7-5421-45c6-8d52-d89e29178151 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:55.672206 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879c17a7-5421-45c6-8d52-d89e29178151 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:55.676246 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9133758-b3ef-45f7-8870-dbae63c95c30 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:55.678413 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9133758-b3ef-45f7-8870-dbae63c95c30 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:55.701005 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application [None req-d9133758-b3ef-45f7-8870-dbae63c95c30 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 18 04:58:55.701005 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:58:55.701005 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:58:55.701005 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:58:55.701005 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:58:55.701005 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:58:55.701005 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:58:55.701005 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:58:55.701005 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:58:55.701005 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:58:55.701005 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:58:55.701005 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:58:55.701005 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 18 04:58:55.701005 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 18 04:58:55.701005 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 18 04:58:55.701947 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 18 04:58:55.701947 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:58:55.701947 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:58:55.701947 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:58:55.701947 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:58:55.701947 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:58:55.701947 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:58:55.701947 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:58:55.701947 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application with Lock( Jun 18 04:58:55.701947 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:58:55.701947 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:58:55.701947 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:58:55.701947 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:58:55.701947 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:58:55.701947 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:58:55.701947 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:58:55.701947 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:58:55.702806 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 18 04:58:55.702806 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 18 04:58:55.702806 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 18 04:58:55.702806 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 18 04:58:55.702806 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 18 04:58:55.702806 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 18 04:58:55.702806 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 18 04:58:55.702806 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 04:58:55.702806 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 259/532] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Jun 18 04:58:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:58:55.702806 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9133758-b3ef-45f7-8870-dbae63c95c30 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:55.730926 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:55.786758 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 274/533] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Jun 18 04:58:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 18 04:58:55.788192 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879c17a7-5421-45c6-8d52-d89e29178151 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:55.809538 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:55.818267 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9133758-b3ef-45f7-8870-dbae63c95c30 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:55.865314 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da637aea-1c80-4609-a0d0-ed859bb14588 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:55.866591 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da637aea-1c80-4609-a0d0-ed859bb14588 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:55.866877 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da637aea-1c80-4609-a0d0-ed859bb14588 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:55.867156 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da637aea-1c80-4609-a0d0-ed859bb14588 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:55.868098 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da637aea-1c80-4609-a0d0-ed859bb14588 None admin] RBAC: Adding query filter params (role.id=5ec7b68b793d4a54abd30460961346b9, scope.project.id=c95f8f3084c5472cb8364b7c57207394, user.id=4638e5d1fa124667b07054a87c2d353e) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:58:55.868419 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da637aea-1c80-4609-a0d0-ed859bb14588 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:55.869988 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da637aea-1c80-4609-a0d0-ed859bb14588 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:55.881407 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 260/534] 10.222.0.23 () {56 vars in 1374 bytes} [Sat Jun 18 04:58:55 2022] GET /identity/v3/role_assignments?user.id=4638e5d1fa124667b07054a87c2d353e&scope.project.id=c95f8f3084c5472cb8364b7c57207394&role.id=5ec7b68b793d4a54abd30460961346b9 => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:55.881407 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da637aea-1c80-4609-a0d0-ed859bb14588 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:55.922785 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879c17a7-5421-45c6-8d52-d89e29178151 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:55.981857 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:55.986017 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b20ca5-f4df-41ab-a3a8-a77556306a10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:55.986911 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b20ca5-f4df-41ab-a3a8-a77556306a10 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:55.987419 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b20ca5-f4df-41ab-a3a8-a77556306a10 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:55.987863 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b20ca5-f4df-41ab-a3a8-a77556306a10 None admin] PATH_INFO: `/v3/projects` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:55.988864 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b20ca5-f4df-41ab-a3a8-a77556306a10 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:58:55.989345 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b20ca5-f4df-41ab-a3a8-a77556306a10 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:55.994364 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da637aea-1c80-4609-a0d0-ed859bb14588 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:55.994470 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b20ca5-f4df-41ab-a3a8-a77556306a10 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:56.031338 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d2f2ff-77d0-4d70-a583-e21d4a9b9324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:56.032139 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d2f2ff-77d0-4d70-a583-e21d4a9b9324 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:56.032427 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d2f2ff-77d0-4d70-a583-e21d4a9b9324 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:56.032703 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d2f2ff-77d0-4d70-a583-e21d4a9b9324 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:56.039888 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 275/535] 10.222.0.23 () {56 vars in 1148 bytes} [Sat Jun 18 04:58:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:56.045756 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b20ca5-f4df-41ab-a3a8-a77556306a10 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:56.045929 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d2f2ff-77d0-4d70-a583-e21d4a9b9324 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:56.047338 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d2f2ff-77d0-4d70-a583-e21d4a9b9324 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:56.069857 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application [None req-b4d2f2ff-77d0-4d70-a583-e21d4a9b9324 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 04:58:56.069857 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:58:56.069857 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:58:56.069857 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:58:56.069857 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:58:56.069857 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:58:56.069857 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:58:56.069857 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:58:56.069857 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:58:56.069857 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:58:56.069857 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:58:56.069857 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:58:56.069857 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 18 04:58:56.069857 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 18 04:58:56.069857 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 18 04:58:56.070834 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 18 04:58:56.070834 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:58:56.070834 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:58:56.070834 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:58:56.070834 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:58:56.070834 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:58:56.070834 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:58:56.070834 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:58:56.070834 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application with Lock( Jun 18 04:58:56.070834 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:58:56.070834 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:58:56.070834 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:58:56.070834 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:58:56.070834 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:58:56.070834 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:58:56.070834 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:58:56.070834 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:58:56.071665 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 18 04:58:56.071665 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 18 04:58:56.071665 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 04:58:56.071665 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 04:58:56.071665 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 261/536] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Jun 18 04:58:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:58:56.077035 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d2f2ff-77d0-4d70-a583-e21d4a9b9324 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:56.180540 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:56.206970 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b20ca5-f4df-41ab-a3a8-a77556306a10 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:56.218461 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d2f2ff-77d0-4d70-a583-e21d4a9b9324 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:56.258505 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1d7918-413f-4dac-951b-5777b60bb633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:56.259101 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1d7918-413f-4dac-951b-5777b60bb633 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:56.259388 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1d7918-413f-4dac-951b-5777b60bb633 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:56.259663 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1d7918-413f-4dac-951b-5777b60bb633 None admin] PATH_INFO: `/v3/domains` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:56.260394 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1d7918-413f-4dac-951b-5777b60bb633 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:58:56.260705 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1d7918-413f-4dac-951b-5777b60bb633 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:56.262103 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1d7918-413f-4dac-951b-5777b60bb633 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:56.263113 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8ad21f-5719-4a17-93db-9d71bfce56cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:56.264110 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8ad21f-5719-4a17-93db-9d71bfce56cb None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:56.264692 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8ad21f-5719-4a17-93db-9d71bfce56cb None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:56.265159 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8ad21f-5719-4a17-93db-9d71bfce56cb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:56.266211 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8ad21f-5719-4a17-93db-9d71bfce56cb None admin] RBAC: Adding query filter params (role.id=5ec7b68b793d4a54abd30460961346b9, scope.project.id=c95f8f3084c5472cb8364b7c57207394, user.id=d9de0428abc8400aa338aba4f7c34c94) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:58:56.266739 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8ad21f-5719-4a17-93db-9d71bfce56cb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:56.268376 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8ad21f-5719-4a17-93db-9d71bfce56cb None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:56.281921 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 262/537] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Jun 18 04:58:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:56.281921 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1d7918-413f-4dac-951b-5777b60bb633 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:56.288013 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 276/538] 10.222.0.23 () {56 vars in 1374 bytes} [Sat Jun 18 04:58:56 2022] GET /identity/v3/role_assignments?user.id=d9de0428abc8400aa338aba4f7c34c94&scope.project.id=c95f8f3084c5472cb8364b7c57207394&role.id=5ec7b68b793d4a54abd30460961346b9 => generated 259 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:56.288603 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:56.403262 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1d7918-413f-4dac-951b-5777b60bb633 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:56.441133 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d10090d-e103-4a08-85fe-578439d57b99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:56.441964 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d10090d-e103-4a08-85fe-578439d57b99 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:56.442253 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d10090d-e103-4a08-85fe-578439d57b99 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:56.442534 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d10090d-e103-4a08-85fe-578439d57b99 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:56.450538 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d10090d-e103-4a08-85fe-578439d57b99 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:56.452019 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d10090d-e103-4a08-85fe-578439d57b99 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:56.467889 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application [None req-7d10090d-e103-4a08-85fe-578439d57b99 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 18 04:58:56.467889 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:58:56.467889 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:58:56.467889 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:58:56.467889 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:58:56.467889 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:58:56.467889 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:58:56.467889 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:58:56.467889 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:58:56.467889 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:58:56.467889 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:58:56.467889 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:58:56.467889 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 18 04:58:56.467889 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 18 04:58:56.467889 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 18 04:58:56.468783 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 18 04:58:56.468783 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:58:56.468783 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:58:56.468783 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:58:56.468783 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:58:56.468783 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:58:56.468783 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:58:56.468783 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:58:56.468783 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application with Lock( Jun 18 04:58:56.468783 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:58:56.468783 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:58:56.468783 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:58:56.468783 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:58:56.468783 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:58:56.468783 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:58:56.468783 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:58:56.468783 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:58:56.469631 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 18 04:58:56.469631 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 18 04:58:56.469631 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 18 04:58:56.469631 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 18 04:58:56.469631 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 18 04:58:56.469631 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 18 04:58:56.469631 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 18 04:58:56.469631 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 04:58:56.469631 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 263/539] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Jun 18 04:58:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 18 04:58:56.478559 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8ad21f-5719-4a17-93db-9d71bfce56cb None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:56.618265 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8ad21f-5719-4a17-93db-9d71bfce56cb None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:56.664444 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62b7d10-60fe-4198-af5b-0b8cbaaefd44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:56.665118 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62b7d10-60fe-4198-af5b-0b8cbaaefd44 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:56.665422 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62b7d10-60fe-4198-af5b-0b8cbaaefd44 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:56.665708 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62b7d10-60fe-4198-af5b-0b8cbaaefd44 None admin] PATH_INFO: `/v3/projects` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:56.666497 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62b7d10-60fe-4198-af5b-0b8cbaaefd44 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:58:56.666809 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62b7d10-60fe-4198-af5b-0b8cbaaefd44 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:56.668260 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62b7d10-60fe-4198-af5b-0b8cbaaefd44 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:56.688024 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 277/540] 10.222.0.23 () {56 vars in 1148 bytes} [Sat Jun 18 04:58:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:56.694215 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d10090d-e103-4a08-85fe-578439d57b99 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:56.776235 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98387a98-7b95-4809-a3f8-b7a7369c6e89 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:56.776550 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98387a98-7b95-4809-a3f8-b7a7369c6e89 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:56.777141 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98387a98-7b95-4809-a3f8-b7a7369c6e89 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:56.778953 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:56.779062 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 278/541] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 04:58:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 18 04:58:56.783802 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201b9223-506e-4596-b4d7-38b18946d33c None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:56.784088 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201b9223-506e-4596-b4d7-38b18946d33c None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:56.784369 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201b9223-506e-4596-b4d7-38b18946d33c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:56.800980 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:56.824322 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d10090d-e103-4a08-85fe-578439d57b99 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:56.830521 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-201b9223-506e-4596-b4d7-38b18946d33c None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 04:58:56.831759 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-201b9223-506e-4596-b4d7-38b18946d33c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 04:58:56.862689 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7a2fd8-97d0-484a-858a-f9e6508da8a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:56.863356 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7a2fd8-97d0-484a-858a-f9e6508da8a5 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:56.864041 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7a2fd8-97d0-484a-858a-f9e6508da8a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:56.864335 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7a2fd8-97d0-484a-858a-f9e6508da8a5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:56.865568 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7a2fd8-97d0-484a-858a-f9e6508da8a5 None admin] RBAC: Adding query filter params (role.id=5ec7b68b793d4a54abd30460961346b9, scope.project.id=c95f8f3084c5472cb8364b7c57207394, user.id=e78c4d7cde8f4ad9aa479bb09e325db3) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:58:56.865895 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7a2fd8-97d0-484a-858a-f9e6508da8a5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:56.867558 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7a2fd8-97d0-484a-858a-f9e6508da8a5 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:56.874616 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 264/542] 10.222.0.23 () {56 vars in 1374 bytes} [Sat Jun 18 04:58:56 2022] GET /identity/v3/role_assignments?user.id=e78c4d7cde8f4ad9aa479bb09e325db3&scope.project.id=c95f8f3084c5472cb8364b7c57207394&role.id=5ec7b68b793d4a54abd30460961346b9 => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:56.938711 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 279/543] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 04:58:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 18 04:58:56.975133 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7a2fd8-97d0-484a-858a-f9e6508da8a5 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:57.106116 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7a2fd8-97d0-484a-858a-f9e6508da8a5 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:57.133848 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:57.150099 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe059130-ec4c-4e6e-8d99-b76fd5a29b90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:57.150900 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe059130-ec4c-4e6e-8d99-b76fd5a29b90 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:57.151191 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe059130-ec4c-4e6e-8d99-b76fd5a29b90 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:57.151470 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe059130-ec4c-4e6e-8d99-b76fd5a29b90 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:57.162415 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe059130-ec4c-4e6e-8d99-b76fd5a29b90 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 18 04:58:57.170456 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application [None req-fe059130-ec4c-4e6e-8d99-b76fd5a29b90 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 18 04:58:57.170456 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:58:57.170456 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:58:57.170456 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:58:57.170456 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:58:57.170456 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:58:57.170456 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:58:57.170456 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:58:57.170456 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:58:57.170456 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:58:57.170456 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:58:57.170456 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:58:57.170456 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 18 04:58:57.170456 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 18 04:58:57.170456 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 18 04:58:57.170456 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 18 04:58:57.171517 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 18 04:58:57.171517 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 18 04:58:57.171517 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 18 04:58:57.171517 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 18 04:58:57.171517 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:58:57.171517 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:58:57.171517 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:58:57.171517 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:58:57.171517 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:58:57.171517 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:58:57.171517 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:58:57.171517 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application with Lock( Jun 18 04:58:57.171517 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:58:57.171517 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:58:57.171517 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:58:57.171517 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:58:57.172470 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:58:57.172470 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:58:57.172470 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:58:57.172470 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:58:57.172470 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 18 04:58:57.172470 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 18 04:58:57.172470 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 18 04:58:57.172470 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 18 04:58:57.172470 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 18 04:58:57.172470 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 18 04:58:57.172470 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 18 04:58:57.172470 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 04:58:57.175095 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 265/544] 10.222.0.23 () {56 vars in 1102 bytes} [Sat Jun 18 04:58:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:58:57.181374 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201b9223-506e-4596-b4d7-38b18946d33c None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:57.329047 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201b9223-506e-4596-b4d7-38b18946d33c None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:57.366878 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:57.372229 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42ae741-0b38-4d36-b36f-5cb015fccf80 None None] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:57.372683 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42ae741-0b38-4d36-b36f-5cb015fccf80 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:57.372799 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42ae741-0b38-4d36-b36f-5cb015fccf80 None None] PATH_INFO: `/` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:57.372884 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e732037-f176-4682-9e95-76b7716ead16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:57.373928 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e732037-f176-4682-9e95-76b7716ead16 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:57.374211 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e732037-f176-4682-9e95-76b7716ead16 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:57.374485 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e732037-f176-4682-9e95-76b7716ead16 None admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:57.375237 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e732037-f176-4682-9e95-76b7716ead16 None admin] RBAC: Adding query filter params (name=service) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:58:57.375541 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e732037-f176-4682-9e95-76b7716ead16 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:57.376921 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e732037-f176-4682-9e95-76b7716ead16 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:57.385133 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 266/545] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 04:58:57 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 04:58:57.385335 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10df4299-c677-4d50-bf40-0646540c8a9a None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:57.389446 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 280/546] 10.222.0.23 () {56 vars in 1103 bytes} [Sat Jun 18 04:58:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:57.390483 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10df4299-c677-4d50-bf40-0646540c8a9a None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:57.390776 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10df4299-c677-4d50-bf40-0646540c8a9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:57.396350 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e732037-f176-4682-9e95-76b7716ead16 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:57.431865 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-10df4299-c677-4d50-bf40-0646540c8a9a None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 04:58:57.433067 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-10df4299-c677-4d50-bf40-0646540c8a9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 04:58:57.571120 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 267/547] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 04:58:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 18 04:58:57.577907 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e732037-f176-4682-9e95-76b7716ead16 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:57.612736 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10df4299-c677-4d50-bf40-0646540c8a9a None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:57.638900 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dee7e35-cb87-4328-8b70-42b5f4ac9125 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:57.639318 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dee7e35-cb87-4328-8b70-42b5f4ac9125 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:57.639740 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dee7e35-cb87-4328-8b70-42b5f4ac9125 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:57.639915 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dee7e35-cb87-4328-8b70-42b5f4ac9125 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:57.657559 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dee7e35-cb87-4328-8b70-42b5f4ac9125 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:57.657559 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dee7e35-cb87-4328-8b70-42b5f4ac9125 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:57.674592 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-7dee7e35-cb87-4328-8b70-42b5f4ac9125 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 04:58:57.674592 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:58:57.674592 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:58:57.674592 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:58:57.674592 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:58:57.674592 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:58:57.674592 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:58:57.674592 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:58:57.674592 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:58:57.674592 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:58:57.674592 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:58:57.674592 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:58:57.674592 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 18 04:58:57.674592 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 18 04:58:57.674592 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 18 04:58:57.675598 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 18 04:58:57.675598 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:58:57.675598 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:58:57.675598 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:58:57.675598 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:58:57.675598 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:58:57.675598 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:58:57.675598 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:58:57.675598 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jun 18 04:58:57.675598 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:58:57.675598 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:58:57.675598 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:58:57.675598 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:58:57.675598 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:58:57.675598 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:58:57.675598 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:58:57.675598 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:58:57.676868 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 18 04:58:57.676868 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 18 04:58:57.676868 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 04:58:57.676868 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 04:58:57.676868 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 281/548] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Jun 18 04:58:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 18 04:58:57.682416 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dee7e35-cb87-4328-8b70-42b5f4ac9125 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:57.741813 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:57.788272 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10df4299-c677-4d50-bf40-0646540c8a9a None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:57.806702 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:57.831852 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880453d3-ddf7-42cd-b2c8-a1c8a94d1bdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:57.832670 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880453d3-ddf7-42cd-b2c8-a1c8a94d1bdf None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:57.832956 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880453d3-ddf7-42cd-b2c8-a1c8a94d1bdf None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:57.833234 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880453d3-ddf7-42cd-b2c8-a1c8a94d1bdf None admin] PATH_INFO: `/v3/roles/service` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:57.851821 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880453d3-ddf7-42cd-b2c8-a1c8a94d1bdf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 18 04:58:57.855551 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dee7e35-cb87-4328-8b70-42b5f4ac9125 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:57.859905 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application [None req-880453d3-ddf7-42cd-b2c8-a1c8a94d1bdf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 18 04:58:57.859905 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:58:57.859905 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:58:57.859905 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:58:57.859905 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:58:57.859905 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:58:57.859905 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:58:57.859905 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:58:57.859905 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:58:57.859905 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:58:57.859905 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:58:57.859905 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:58:57.859905 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 18 04:58:57.859905 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 18 04:58:57.859905 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 18 04:58:57.859905 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 18 04:58:57.860959 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 18 04:58:57.860959 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 18 04:58:57.860959 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 18 04:58:57.860959 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 18 04:58:57.860959 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:58:57.860959 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:58:57.860959 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:58:57.860959 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:58:57.860959 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:58:57.860959 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:58:57.860959 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:58:57.860959 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application with Lock( Jun 18 04:58:57.860959 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:58:57.860959 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:58:57.860959 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:58:57.860959 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:58:57.862181 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:58:57.862181 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:58:57.862181 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:58:57.862181 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:58:57.862181 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 18 04:58:57.862181 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 18 04:58:57.862181 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 18 04:58:57.862181 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 18 04:58:57.862181 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 18 04:58:57.862181 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 18 04:58:57.862181 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 18 04:58:57.862181 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 04:58:57.862181 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 268/549] 10.222.0.23 () {56 vars in 1102 bytes} [Sat Jun 18 04:58:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 18 04:58:57.868517 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880453d3-ddf7-42cd-b2c8-a1c8a94d1bdf None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:57.954081 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469ca3c2-7d97-45cf-a655-28fdf1ee09e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:57.954752 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469ca3c2-7d97-45cf-a655-28fdf1ee09e6 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:57.955037 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469ca3c2-7d97-45cf-a655-28fdf1ee09e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:57.955317 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469ca3c2-7d97-45cf-a655-28fdf1ee09e6 None admin] PATH_INFO: `/v3/domains` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:57.956036 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469ca3c2-7d97-45cf-a655-28fdf1ee09e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:58:57.956344 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469ca3c2-7d97-45cf-a655-28fdf1ee09e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:57.957756 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469ca3c2-7d97-45cf-a655-28fdf1ee09e6 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:57.981426 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880453d3-ddf7-42cd-b2c8-a1c8a94d1bdf None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:57.984409 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 282/550] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Jun 18 04:58:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 18 04:58:57.989867 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469ca3c2-7d97-45cf-a655-28fdf1ee09e6 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:58.036791 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aede3cac-23a9-41a4-9b19-9e4cbd456b40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:58.037689 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aede3cac-23a9-41a4-9b19-9e4cbd456b40 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:58.037968 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aede3cac-23a9-41a4-9b19-9e4cbd456b40 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:58.038245 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aede3cac-23a9-41a4-9b19-9e4cbd456b40 None admin] PATH_INFO: `/v3/roles` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:58.038991 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aede3cac-23a9-41a4-9b19-9e4cbd456b40 None admin] RBAC: Adding query filter params (name=service) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:58:58.039303 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aede3cac-23a9-41a4-9b19-9e4cbd456b40 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:58.040681 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aede3cac-23a9-41a4-9b19-9e4cbd456b40 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:58.050080 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 269/551] 10.222.0.23 () {56 vars in 1103 bytes} [Sat Jun 18 04:58:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 18 04:58:58.056033 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aede3cac-23a9-41a4-9b19-9e4cbd456b40 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:58.132588 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:58.168170 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469ca3c2-7d97-45cf-a655-28fdf1ee09e6 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:58.175829 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:58.204973 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00406f6-d99c-4808-952e-f2d5e28d0520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:58.206483 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00406f6-d99c-4808-952e-f2d5e28d0520 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:58.206907 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00406f6-d99c-4808-952e-f2d5e28d0520 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:58.207347 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00406f6-d99c-4808-952e-f2d5e28d0520 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:58.216108 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00406f6-d99c-4808-952e-f2d5e28d0520 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:58.217898 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00406f6-d99c-4808-952e-f2d5e28d0520 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:58.238393 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-a00406f6-d99c-4808-952e-f2d5e28d0520 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 18 04:58:58.238393 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:58:58.238393 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:58:58.238393 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:58:58.238393 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:58:58.238393 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:58:58.238393 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:58:58.238393 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:58:58.238393 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:58:58.238393 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:58:58.238393 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:58:58.238393 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:58:58.238393 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 18 04:58:58.238393 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 18 04:58:58.238393 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 18 04:58:58.238393 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 18 04:58:58.239367 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:58:58.239367 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:58:58.239367 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 18 04:58:58.239367 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 18 04:58:58.239367 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 18 04:58:58.239367 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 18 04:58:58.239367 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:58:58.239367 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:58:58.239367 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:58:58.239367 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:58:58.239367 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:58:58.239367 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jun 18 04:58:58.239367 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:58:58.239367 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:58:58.239367 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:58:58.239367 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:58:58.239367 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:58:58.239367 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:58:58.241108 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:58:58.241108 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:58:58.241108 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 18 04:58:58.241108 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 18 04:58:58.241108 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 18 04:58:58.241108 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 18 04:58:58.241108 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 18 04:58:58.241108 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 18 04:58:58.241108 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 18 04:58:58.241108 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 04:58:58.243979 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 283/552] 10.222.0.23 () {56 vars in 1096 bytes} [Sat Jun 18 04:58:57 2022] GET /identity/v3/users/swift => generated 83 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 18 04:58:58.249576 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00406f6-d99c-4808-952e-f2d5e28d0520 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:58.259363 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aede3cac-23a9-41a4-9b19-9e4cbd456b40 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:58.372968 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00406f6-d99c-4808-952e-f2d5e28d0520 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:58.398110 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac77b760-8453-4208-bb18-196c7fa0d1dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:58.398921 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac77b760-8453-4208-bb18-196c7fa0d1dc None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:58.399210 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac77b760-8453-4208-bb18-196c7fa0d1dc None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:58.399572 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac77b760-8453-4208-bb18-196c7fa0d1dc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:58.421714 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fb755f-d492-4e2f-9f06-be2e5b1ee163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:58.422392 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fb755f-d492-4e2f-9f06-be2e5b1ee163 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:58.422675 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fb755f-d492-4e2f-9f06-be2e5b1ee163 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:58.422950 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fb755f-d492-4e2f-9f06-be2e5b1ee163 None admin] PATH_INFO: `/v3/users` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:58.423793 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fb755f-d492-4e2f-9f06-be2e5b1ee163 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:58:58.424110 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fb755f-d492-4e2f-9f06-be2e5b1ee163 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:58.425589 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fb755f-d492-4e2f-9f06-be2e5b1ee163 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:58.430364 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac77b760-8453-4208-bb18-196c7fa0d1dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:58.431864 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac77b760-8453-4208-bb18-196c7fa0d1dc None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:58.457159 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application [None req-ac77b760-8453-4208-bb18-196c7fa0d1dc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 04:58:58.457159 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:58:58.457159 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:58:58.457159 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:58:58.457159 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:58:58.457159 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:58:58.457159 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:58:58.457159 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:58:58.457159 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:58:58.457159 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:58:58.457159 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:58:58.457159 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:58:58.457159 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 18 04:58:58.457159 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 18 04:58:58.457159 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 18 04:58:58.458072 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 18 04:58:58.458072 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:58:58.458072 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:58:58.458072 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:58:58.458072 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:58:58.458072 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:58:58.458072 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:58:58.458072 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:58:58.458072 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application with Lock( Jun 18 04:58:58.458072 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:58:58.458072 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:58:58.458072 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:58:58.458072 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:58:58.458072 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:58:58.458072 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:58:58.458072 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:58:58.458072 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:58:58.459447 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 18 04:58:58.459447 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 18 04:58:58.459447 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 04:58:58.459447 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 04:58:58.459447 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 270/553] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Jun 18 04:58:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 403 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:58:58.464568 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac77b760-8453-4208-bb18-196c7fa0d1dc None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:58.470267 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 284/554] 10.222.0.23 () {56 vars in 1135 bytes} [Sat Jun 18 04:58:58 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:58.475595 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fb755f-d492-4e2f-9f06-be2e5b1ee163 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:58.510939 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:58.582287 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:58.598079 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac77b760-8453-4208-bb18-196c7fa0d1dc None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:58.634713 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b921fb1-8a1c-4567-8559-febc207949dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:58.635388 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b921fb1-8a1c-4567-8559-febc207949dd None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:58.635674 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b921fb1-8a1c-4567-8559-febc207949dd None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:58.635953 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b921fb1-8a1c-4567-8559-febc207949dd None admin] PATH_INFO: `/v3/domains` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:58.636688 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b921fb1-8a1c-4567-8559-febc207949dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:58:58.637005 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b921fb1-8a1c-4567-8559-febc207949dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:58.638417 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b921fb1-8a1c-4567-8559-febc207949dd None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:58.654651 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fb755f-d492-4e2f-9f06-be2e5b1ee163 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:58.660014 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 271/555] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Jun 18 04:58:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:58.668268 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3602ebc4-2e59-4eeb-8627-39d0f5e81ccb None None] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:58.668563 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3602ebc4-2e59-4eeb-8627-39d0f5e81ccb None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:58.668853 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3602ebc4-2e59-4eeb-8627-39d0f5e81ccb None None] PATH_INFO: `/` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:58.670899 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 272/556] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 04:58:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 04:58:58.671770 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3602ebc4-2e59-4eeb-8627-39d0f5e81ccb None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:58.692326 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22716210-df7e-4493-b419-0ca9cfebfe87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:58.692976 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22716210-df7e-4493-b419-0ca9cfebfe87 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:58.693848 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22716210-df7e-4493-b419-0ca9cfebfe87 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:58.694132 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22716210-df7e-4493-b419-0ca9cfebfe87 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:58.801729 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22716210-df7e-4493-b419-0ca9cfebfe87 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:58.804350 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22716210-df7e-4493-b419-0ca9cfebfe87 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:58.806411 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3602ebc4-2e59-4eeb-8627-39d0f5e81ccb None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:58.831004 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:58.852177 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabce337-8828-47c9-8185-841b3872d3ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:58.853111 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabce337-8828-47c9-8185-841b3872d3ea None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:58.853546 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabce337-8828-47c9-8185-841b3872d3ea None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:58.853849 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabce337-8828-47c9-8185-841b3872d3ea None admin] PATH_INFO: `/v3/users/nova` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:58.862662 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabce337-8828-47c9-8185-841b3872d3ea None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:58.864557 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabce337-8828-47c9-8185-841b3872d3ea None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:58.885339 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application [None req-dabce337-8828-47c9-8185-841b3872d3ea None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 18 04:58:58.885339 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:58:58.885339 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:58:58.885339 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:58:58.885339 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:58:58.885339 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:58:58.885339 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:58:58.885339 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:58:58.885339 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:58:58.885339 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:58:58.885339 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:58:58.885339 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:58:58.885339 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 18 04:58:58.885339 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 18 04:58:58.885339 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 18 04:58:58.885339 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 18 04:58:58.886776 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:58:58.886776 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:58:58.886776 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 18 04:58:58.886776 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 18 04:58:58.886776 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 18 04:58:58.886776 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 18 04:58:58.886776 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:58:58.886776 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:58:58.886776 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:58:58.886776 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:58:58.886776 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:58:58.886776 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application with Lock( Jun 18 04:58:58.886776 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:58:58.886776 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:58:58.886776 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:58:58.886776 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:58:58.886776 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:58:58.886776 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:58:58.887702 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:58:58.887702 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:58:58.887702 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 18 04:58:58.887702 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 18 04:58:58.887702 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 18 04:58:58.887702 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 18 04:58:58.887702 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 18 04:58:58.887702 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 18 04:58:58.887702 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 18 04:58:58.887702 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 04:58:58.887702 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 273/557] 10.222.0.23 () {56 vars in 1093 bytes} [Sat Jun 18 04:58:58 2022] GET /identity/v3/users/nova => generated 82 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:58:58.888284 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d479b16-71da-41cb-8877-f95038c98a84 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:58.888357 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d479b16-71da-41cb-8877-f95038c98a84 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:58.888643 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d479b16-71da-41cb-8877-f95038c98a84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:58.909268 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-2d479b16-71da-41cb-8877-f95038c98a84 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 04:58:58.910688 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-2d479b16-71da-41cb-8877-f95038c98a84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 04:58:58.915652 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:58.922964 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 285/558] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Jun 18 04:58:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 18 04:58:58.928306 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22716210-df7e-4493-b419-0ca9cfebfe87 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:59.013574 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 274/559] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 04:58:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 18 04:58:59.013574 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d479b16-71da-41cb-8877-f95038c98a84 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:59.075414 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22716210-df7e-4493-b419-0ca9cfebfe87 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:59.099821 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:59.110283 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6cd67b-ebe7-4bb9-86cd-e662ac3d99c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:59.110969 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6cd67b-ebe7-4bb9-86cd-e662ac3d99c4 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:59.111230 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6cd67b-ebe7-4bb9-86cd-e662ac3d99c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:59.111507 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6cd67b-ebe7-4bb9-86cd-e662ac3d99c4 None admin] PATH_INFO: `/v3/users` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:59.112336 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6cd67b-ebe7-4bb9-86cd-e662ac3d99c4 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:58:59.112652 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6cd67b-ebe7-4bb9-86cd-e662ac3d99c4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:59.114147 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6cd67b-ebe7-4bb9-86cd-e662ac3d99c4 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:59.136751 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 286/560] 10.222.0.23 () {56 vars in 1133 bytes} [Sat Jun 18 04:58:58 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:59.137663 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6cd67b-ebe7-4bb9-86cd-e662ac3d99c4 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:59.149052 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d479b16-71da-41cb-8877-f95038c98a84 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:59.185108 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48498a89-4e7d-435c-a463-8b979b2d8928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:59.185929 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48498a89-4e7d-435c-a463-8b979b2d8928 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:59.186208 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48498a89-4e7d-435c-a463-8b979b2d8928 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:59.186482 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48498a89-4e7d-435c-a463-8b979b2d8928 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:59.195932 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:59.198402 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48498a89-4e7d-435c-a463-8b979b2d8928 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:59.199875 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48498a89-4e7d-435c-a463-8b979b2d8928 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:59.217847 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application [None req-48498a89-4e7d-435c-a463-8b979b2d8928 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 04:58:59.217847 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:58:59.217847 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:58:59.217847 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:58:59.217847 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:58:59.217847 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:58:59.217847 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:58:59.217847 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:58:59.217847 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:58:59.217847 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:58:59.217847 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:58:59.217847 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:58:59.217847 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 18 04:58:59.217847 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 18 04:58:59.217847 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 18 04:58:59.218782 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 18 04:58:59.218782 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:58:59.218782 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:58:59.218782 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:58:59.218782 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:58:59.218782 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:58:59.218782 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:58:59.218782 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:58:59.218782 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application with Lock( Jun 18 04:58:59.218782 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:58:59.218782 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:58:59.218782 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:58:59.218782 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:58:59.218782 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:58:59.218782 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:58:59.218782 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:58:59.218782 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:58:59.219770 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 18 04:58:59.219770 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 18 04:58:59.219770 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 04:58:59.219770 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 04:58:59.219770 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 275/561] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Jun 18 04:58:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:58:59.220766 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9d4ceb-b339-4288-b598-ba551fa27fb4 None None] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:59.220766 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9d4ceb-b339-4288-b598-ba551fa27fb4 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:59.220766 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9d4ceb-b339-4288-b598-ba551fa27fb4 None None] PATH_INFO: `/` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:59.224585 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 276/562] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 04:58:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 04:58:59.224585 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9d4ceb-b339-4288-b598-ba551fa27fb4 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:59.246357 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6cd67b-ebe7-4bb9-86cd-e662ac3d99c4 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:59.280702 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0826c510-f897-4568-a33a-76381d367c3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:59.281642 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0826c510-f897-4568-a33a-76381d367c3c None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:59.281925 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0826c510-f897-4568-a33a-76381d367c3c None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:59.282205 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0826c510-f897-4568-a33a-76381d367c3c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:59.292556 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0826c510-f897-4568-a33a-76381d367c3c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 18 04:58:59.300224 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-0826c510-f897-4568-a33a-76381d367c3c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 18 04:58:59.300224 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:58:59.300224 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:58:59.300224 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:58:59.300224 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:58:59.300224 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:58:59.300224 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:58:59.300224 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:58:59.300224 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:58:59.300224 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:58:59.300224 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:58:59.300224 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:58:59.300224 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 18 04:58:59.300224 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 18 04:58:59.300224 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 18 04:58:59.300224 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 18 04:58:59.301209 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 18 04:58:59.301209 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 18 04:58:59.301209 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 18 04:58:59.301209 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 18 04:58:59.301209 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:58:59.301209 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:58:59.301209 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:58:59.301209 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:58:59.301209 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:58:59.301209 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:58:59.301209 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:58:59.301209 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jun 18 04:58:59.301209 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:58:59.301209 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:58:59.301209 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:58:59.301209 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:58:59.302099 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:58:59.302099 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:58:59.302099 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:58:59.302099 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:58:59.302099 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 18 04:58:59.302099 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 18 04:58:59.302099 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 18 04:58:59.302099 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 18 04:58:59.302099 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 18 04:58:59.302099 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 18 04:58:59.302099 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 18 04:58:59.302099 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 04:58:59.302099 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 287/563] 10.222.0.23 () {56 vars in 1102 bytes} [Sat Jun 18 04:58:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:58:59.303300 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0826c510-f897-4568-a33a-76381d367c3c None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:59.336602 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9d4ceb-b339-4288-b598-ba551fa27fb4 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:59.345721 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:59.371919 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71669528-7c64-4aac-9878-b0d63021f20d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:59.372697 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71669528-7c64-4aac-9878-b0d63021f20d None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:59.372969 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71669528-7c64-4aac-9878-b0d63021f20d None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:59.373230 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71669528-7c64-4aac-9878-b0d63021f20d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:59.384357 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71669528-7c64-4aac-9878-b0d63021f20d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:59.388428 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71669528-7c64-4aac-9878-b0d63021f20d None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:59.403774 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application [None req-71669528-7c64-4aac-9878-b0d63021f20d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 04:58:59.403774 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:58:59.403774 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:58:59.403774 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:58:59.403774 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:58:59.403774 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:58:59.403774 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:58:59.403774 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:58:59.403774 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:58:59.403774 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:58:59.403774 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:58:59.403774 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:58:59.403774 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 18 04:58:59.403774 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 18 04:58:59.403774 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 18 04:58:59.405212 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 18 04:58:59.405212 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:58:59.405212 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:58:59.405212 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:58:59.405212 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:58:59.405212 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:58:59.405212 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:58:59.405212 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:58:59.405212 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application with Lock( Jun 18 04:58:59.405212 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:58:59.405212 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:58:59.405212 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:58:59.405212 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:58:59.405212 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:58:59.405212 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:58:59.405212 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:58:59.405212 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:58:59.406224 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 18 04:58:59.406224 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 18 04:58:59.406224 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 04:58:59.406224 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 04:58:59.406449 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0826c510-f897-4568-a33a-76381d367c3c None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:59.409537 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 277/564] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Jun 18 04:58:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:58:59.410813 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c9928e-6f20-4223-ad37-30e2e6e8ac92 None None] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:59.411102 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c9928e-6f20-4223-ad37-30e2e6e8ac92 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:59.411377 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c9928e-6f20-4223-ad37-30e2e6e8ac92 None None] PATH_INFO: `/` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:59.412920 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 278/565] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 04:58:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 04:58:59.413730 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c6c3be-762a-469f-9279-698de869685f None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:59.414013 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c6c3be-762a-469f-9279-698de869685f None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:59.414288 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c6c3be-762a-469f-9279-698de869685f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:59.430930 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-16c6c3be-762a-469f-9279-698de869685f None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 04:58:59.432112 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-16c6c3be-762a-469f-9279-698de869685f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 04:58:59.440085 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e62a63-fa4e-46be-a029-51e1782dde52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:59.440758 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e62a63-fa4e-46be-a029-51e1782dde52 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:59.441025 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e62a63-fa4e-46be-a029-51e1782dde52 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:59.441302 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e62a63-fa4e-46be-a029-51e1782dde52 None admin] PATH_INFO: `/v3/domains` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:59.442048 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e62a63-fa4e-46be-a029-51e1782dde52 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:58:59.442357 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e62a63-fa4e-46be-a029-51e1782dde52 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:59.444866 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e62a63-fa4e-46be-a029-51e1782dde52 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:59.456346 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 288/566] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Jun 18 04:58:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:59.457114 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e62a63-fa4e-46be-a029-51e1782dde52 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:59.471993 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:59.498695 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 279/567] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 04:58:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 18 04:58:59.499501 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c6c3be-762a-469f-9279-698de869685f None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:59.569524 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e62a63-fa4e-46be-a029-51e1782dde52 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:59.571033 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:59.598430 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c6c3be-762a-469f-9279-698de869685f None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:59.609323 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5594c8-d202-4b0f-bcee-96a78bf7ef17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:59.610011 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5594c8-d202-4b0f-bcee-96a78bf7ef17 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:59.610291 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5594c8-d202-4b0f-bcee-96a78bf7ef17 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:59.610561 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5594c8-d202-4b0f-bcee-96a78bf7ef17 None admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:59.611300 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5594c8-d202-4b0f-bcee-96a78bf7ef17 None admin] RBAC: Adding query filter params (name=service) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:58:59.611603 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5594c8-d202-4b0f-bcee-96a78bf7ef17 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:59.612973 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5594c8-d202-4b0f-bcee-96a78bf7ef17 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:59.621043 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 289/568] 10.222.0.23 () {56 vars in 1103 bytes} [Sat Jun 18 04:58:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:59.622405 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362c6303-eb31-4422-bf62-3f8bfab187cc None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:59.622697 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362c6303-eb31-4422-bf62-3f8bfab187cc None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:59.622974 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362c6303-eb31-4422-bf62-3f8bfab187cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:59.638853 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c4699e-1903-444b-8d26-d54c9fad98e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:59.639514 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c4699e-1903-444b-8d26-d54c9fad98e0 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:59.639780 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-362c6303-eb31-4422-bf62-3f8bfab187cc None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 04:58:59.639895 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c4699e-1903-444b-8d26-d54c9fad98e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:59.640067 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c4699e-1903-444b-8d26-d54c9fad98e0 None admin] PATH_INFO: `/v3/domains` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:59.640783 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c4699e-1903-444b-8d26-d54c9fad98e0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:58:59.640957 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-362c6303-eb31-4422-bf62-3f8bfab187cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 04:58:59.641096 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c4699e-1903-444b-8d26-d54c9fad98e0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:59.642495 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c4699e-1903-444b-8d26-d54c9fad98e0 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:59.654829 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 280/569] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Jun 18 04:58:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:58:59.655722 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c4699e-1903-444b-8d26-d54c9fad98e0 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:59.688715 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:59.713133 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 290/570] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 04:58:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 18 04:58:59.714027 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362c6303-eb31-4422-bf62-3f8bfab187cc None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:59.756329 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c4699e-1903-444b-8d26-d54c9fad98e0 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:59.790196 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f229af46-29eb-43ed-871a-9ad7f4cc788a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:59.790979 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f229af46-29eb-43ed-871a-9ad7f4cc788a None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:59.791276 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f229af46-29eb-43ed-871a-9ad7f4cc788a None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:59.791550 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f229af46-29eb-43ed-871a-9ad7f4cc788a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:59.798124 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f229af46-29eb-43ed-871a-9ad7f4cc788a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:59.799567 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f229af46-29eb-43ed-871a-9ad7f4cc788a None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:59.814127 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application [None req-f229af46-29eb-43ed-871a-9ad7f4cc788a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 18 04:58:59.814127 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:58:59.814127 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:58:59.814127 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:58:59.814127 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:58:59.814127 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:58:59.814127 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:58:59.814127 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:58:59.814127 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:58:59.814127 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:58:59.814127 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:58:59.814127 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:58:59.814127 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 18 04:58:59.814127 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 18 04:58:59.814127 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 18 04:58:59.814992 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 18 04:58:59.814992 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:58:59.814992 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:58:59.814992 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:58:59.814992 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:58:59.814992 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:58:59.814992 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:58:59.814992 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:58:59.814992 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application with Lock( Jun 18 04:58:59.814992 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:58:59.814992 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:58:59.814992 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:58:59.814992 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:58:59.814992 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:58:59.814992 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:58:59.814992 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:58:59.814992 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:58:59.816109 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 18 04:58:59.816109 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 18 04:58:59.816109 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 18 04:58:59.816109 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 18 04:58:59.816109 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 18 04:58:59.816109 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 18 04:58:59.816109 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 18 04:58:59.816109 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 04:58:59.816109 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 281/571] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Jun 18 04:58:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:58:59.816109 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f229af46-29eb-43ed-871a-9ad7f4cc788a None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:59.820189 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:59.826769 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362c6303-eb31-4422-bf62-3f8bfab187cc None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:59.861046 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017ba79d-bec2-480a-8f5f-c10549b7197b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:59.861792 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017ba79d-bec2-480a-8f5f-c10549b7197b None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:59.862081 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017ba79d-bec2-480a-8f5f-c10549b7197b None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:59.862358 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017ba79d-bec2-480a-8f5f-c10549b7197b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:59.873331 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017ba79d-bec2-480a-8f5f-c10549b7197b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 18 04:58:59.880929 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-017ba79d-bec2-480a-8f5f-c10549b7197b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 18 04:58:59.880929 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:58:59.880929 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:58:59.880929 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:58:59.880929 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:58:59.880929 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:58:59.880929 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:58:59.880929 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:58:59.880929 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:58:59.880929 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:58:59.880929 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:58:59.880929 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:58:59.880929 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 18 04:58:59.880929 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 18 04:58:59.880929 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 18 04:58:59.880929 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 18 04:58:59.881981 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 18 04:58:59.881981 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 18 04:58:59.881981 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 18 04:58:59.881981 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 18 04:58:59.881981 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:58:59.881981 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:58:59.881981 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:58:59.881981 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:58:59.881981 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:58:59.881981 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:58:59.881981 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:58:59.881981 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jun 18 04:58:59.881981 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:58:59.881981 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:58:59.881981 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:58:59.881981 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:58:59.882852 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:58:59.882852 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:58:59.882852 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:58:59.882852 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:58:59.882852 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 18 04:58:59.882852 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 18 04:58:59.882852 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 18 04:58:59.882852 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 18 04:58:59.882852 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 18 04:58:59.882852 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 18 04:58:59.882852 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 18 04:58:59.882852 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 04:58:59.882852 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 291/572] 10.222.0.23 () {56 vars in 1102 bytes} [Sat Jun 18 04:58:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:58:59.883561 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017ba79d-bec2-480a-8f5f-c10549b7197b None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:58:59.916507 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:58:59.923797 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f229af46-29eb-43ed-871a-9ad7f4cc788a None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:59.958393 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6277e976-2d86-424f-a872-4cc3554c16cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:58:59.959874 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6277e976-2d86-424f-a872-4cc3554c16cd None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:58:59.960154 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6277e976-2d86-424f-a872-4cc3554c16cd None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:58:59.960433 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6277e976-2d86-424f-a872-4cc3554c16cd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:58:59.970552 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6277e976-2d86-424f-a872-4cc3554c16cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:58:59.971993 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6277e976-2d86-424f-a872-4cc3554c16cd None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:58:59.990503 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application [None req-6277e976-2d86-424f-a872-4cc3554c16cd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 04:58:59.990503 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:58:59.990503 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:58:59.990503 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:58:59.990503 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:58:59.990503 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:58:59.990503 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:58:59.990503 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:58:59.990503 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:58:59.990503 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:58:59.990503 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:58:59.990503 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:58:59.990503 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 18 04:58:59.990503 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 18 04:58:59.990503 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 18 04:58:59.991644 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 18 04:58:59.991644 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:58:59.991644 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:58:59.991644 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:58:59.991644 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:58:59.991644 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:58:59.991644 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:58:59.991644 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:58:59.991644 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application with Lock( Jun 18 04:58:59.991644 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:58:59.991644 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:58:59.991644 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:58:59.991644 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:58:59.991644 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:58:59.991644 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:58:59.991644 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:58:59.991644 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:58:59.992523 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 18 04:58:59.992523 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 18 04:58:59.992523 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 04:58:59.992523 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 04:58:59.992523 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 282/573] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Jun 18 04:58:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:58:59.993282 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017ba79d-bec2-480a-8f5f-c10549b7197b None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:58:59.993360 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6277e976-2d86-424f-a872-4cc3554c16cd None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:00.028268 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3125dd06-b8ac-4409-afd4-e27badd19bf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:00.029055 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3125dd06-b8ac-4409-afd4-e27badd19bf7 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:00.029341 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3125dd06-b8ac-4409-afd4-e27badd19bf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:00.029652 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3125dd06-b8ac-4409-afd4-e27badd19bf7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:00.036199 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3125dd06-b8ac-4409-afd4-e27badd19bf7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:00.038483 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3125dd06-b8ac-4409-afd4-e27badd19bf7 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:00.052538 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-3125dd06-b8ac-4409-afd4-e27badd19bf7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 18 04:59:00.052538 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:00.052538 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:59:00.052538 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:59:00.052538 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:59:00.052538 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:59:00.052538 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:59:00.052538 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:59:00.052538 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:59:00.052538 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:59:00.052538 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:59:00.052538 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:59:00.052538 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 18 04:59:00.052538 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 18 04:59:00.052538 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 18 04:59:00.053574 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 18 04:59:00.053574 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:59:00.053574 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:59:00.053574 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:59:00.053574 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:59:00.053574 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:59:00.053574 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:59:00.053574 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:59:00.053574 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jun 18 04:59:00.053574 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:59:00.053574 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:59:00.053574 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:59:00.053574 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:59:00.053574 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:59:00.053574 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:59:00.053574 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:59:00.053574 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:59:00.059351 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 18 04:59:00.059351 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 18 04:59:00.059351 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 18 04:59:00.059351 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 18 04:59:00.059351 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 18 04:59:00.059351 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 18 04:59:00.059351 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 18 04:59:00.059351 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 04:59:00.059351 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 292/574] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Jun 18 04:58:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:59:00.059351 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3125dd06-b8ac-4409-afd4-e27badd19bf7 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:00.072665 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:00.093170 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6277e976-2d86-424f-a872-4cc3554c16cd None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:00.127908 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f99eae-1b08-4782-b367-72bb8a9e8230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:00.128636 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f99eae-1b08-4782-b367-72bb8a9e8230 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:00.128921 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f99eae-1b08-4782-b367-72bb8a9e8230 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:00.129196 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f99eae-1b08-4782-b367-72bb8a9e8230 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:00.139358 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f99eae-1b08-4782-b367-72bb8a9e8230 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 18 04:59:00.147631 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application [None req-02f99eae-1b08-4782-b367-72bb8a9e8230 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 18 04:59:00.147631 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:00.147631 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:59:00.147631 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:59:00.147631 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:59:00.147631 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:59:00.147631 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:59:00.147631 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:59:00.147631 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:59:00.147631 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:59:00.147631 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:59:00.147631 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:59:00.147631 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 18 04:59:00.147631 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 18 04:59:00.147631 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 18 04:59:00.147631 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 18 04:59:00.148672 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 18 04:59:00.148672 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 18 04:59:00.148672 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 18 04:59:00.148672 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 18 04:59:00.148672 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:59:00.148672 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:59:00.148672 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:59:00.148672 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:59:00.148672 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:59:00.148672 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:59:00.148672 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:59:00.148672 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application with Lock( Jun 18 04:59:00.148672 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:59:00.148672 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:59:00.148672 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:59:00.148672 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:59:00.154679 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:59:00.154679 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:59:00.154679 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:59:00.154679 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:59:00.154679 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 18 04:59:00.154679 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 18 04:59:00.154679 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 18 04:59:00.154679 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 18 04:59:00.154679 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 18 04:59:00.154679 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 18 04:59:00.154679 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 18 04:59:00.154679 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 04:59:00.154679 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 283/575] 10.222.0.23 () {56 vars in 1102 bytes} [Sat Jun 18 04:58:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:59:00.154679 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f99eae-1b08-4782-b367-72bb8a9e8230 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:00.161944 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:00.162184 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3125dd06-b8ac-4409-afd4-e27badd19bf7 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:00.195509 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c7db36-75f5-40e7-b0b7-51b5e99583df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:00.196168 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c7db36-75f5-40e7-b0b7-51b5e99583df None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:00.196445 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c7db36-75f5-40e7-b0b7-51b5e99583df None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:00.196717 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c7db36-75f5-40e7-b0b7-51b5e99583df None admin] PATH_INFO: `/v3/projects` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:00.197526 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c7db36-75f5-40e7-b0b7-51b5e99583df None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:00.197845 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c7db36-75f5-40e7-b0b7-51b5e99583df None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:00.199270 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c7db36-75f5-40e7-b0b7-51b5e99583df None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:00.214876 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 293/576] 10.222.0.23 () {56 vars in 1148 bytes} [Sat Jun 18 04:59:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:00.215686 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c7db36-75f5-40e7-b0b7-51b5e99583df None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:00.258939 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f99eae-1b08-4782-b367-72bb8a9e8230 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:00.289696 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:00.293327 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a4dec1-7715-44c0-942a-7dd13854d123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:00.294014 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a4dec1-7715-44c0-942a-7dd13854d123 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:00.294297 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a4dec1-7715-44c0-942a-7dd13854d123 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:00.294571 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a4dec1-7715-44c0-942a-7dd13854d123 None admin] PATH_INFO: `/v3/roles` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:00.295307 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a4dec1-7715-44c0-942a-7dd13854d123 None admin] RBAC: Adding query filter params (name=service) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:00.295618 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a4dec1-7715-44c0-942a-7dd13854d123 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:00.296970 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a4dec1-7715-44c0-942a-7dd13854d123 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:00.305425 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 284/577] 10.222.0.23 () {56 vars in 1103 bytes} [Sat Jun 18 04:59:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:00.306199 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a4dec1-7715-44c0-942a-7dd13854d123 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:00.318535 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c7db36-75f5-40e7-b0b7-51b5e99583df None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:00.352791 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cc9e43-e38e-489e-a5b2-4a02d1a93784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:00.353484 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cc9e43-e38e-489e-a5b2-4a02d1a93784 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:00.353772 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cc9e43-e38e-489e-a5b2-4a02d1a93784 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:00.354051 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cc9e43-e38e-489e-a5b2-4a02d1a93784 None admin] PATH_INFO: `/v3/domains` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:00.354769 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cc9e43-e38e-489e-a5b2-4a02d1a93784 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:00.355084 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cc9e43-e38e-489e-a5b2-4a02d1a93784 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:00.356630 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cc9e43-e38e-489e-a5b2-4a02d1a93784 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:00.368621 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 294/578] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Jun 18 04:59:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:00.369445 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cc9e43-e38e-489e-a5b2-4a02d1a93784 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:00.379127 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:00.415411 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a4dec1-7715-44c0-942a-7dd13854d123 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:00.449614 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093f0994-5761-4501-be88-62b1bba2d19a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:00.450266 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093f0994-5761-4501-be88-62b1bba2d19a None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:00.450549 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093f0994-5761-4501-be88-62b1bba2d19a None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:00.450825 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093f0994-5761-4501-be88-62b1bba2d19a None admin] PATH_INFO: `/v3/projects` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:00.451597 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093f0994-5761-4501-be88-62b1bba2d19a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:00.451915 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093f0994-5761-4501-be88-62b1bba2d19a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:00.453330 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093f0994-5761-4501-be88-62b1bba2d19a None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:00.465762 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 285/579] 10.222.0.23 () {56 vars in 1148 bytes} [Sat Jun 18 04:59:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:00.466538 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093f0994-5761-4501-be88-62b1bba2d19a None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:00.473110 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cc9e43-e38e-489e-a5b2-4a02d1a93784 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:00.508342 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5e2ecd-c59e-4555-8ce2-06d0955c93ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:00.509089 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5e2ecd-c59e-4555-8ce2-06d0955c93ae None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:00.509481 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5e2ecd-c59e-4555-8ce2-06d0955c93ae None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:00.509835 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5e2ecd-c59e-4555-8ce2-06d0955c93ae None admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:00.510650 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5e2ecd-c59e-4555-8ce2-06d0955c93ae None admin] RBAC: Adding query filter params (name=service) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:00.511033 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5e2ecd-c59e-4555-8ce2-06d0955c93ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:00.512485 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5e2ecd-c59e-4555-8ce2-06d0955c93ae None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:00.513972 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:00.521043 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 295/580] 10.222.0.23 () {56 vars in 1103 bytes} [Sat Jun 18 04:59:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:00.522311 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5e2ecd-c59e-4555-8ce2-06d0955c93ae None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:00.566293 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093f0994-5761-4501-be88-62b1bba2d19a None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:00.594101 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:00.602067 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8ea05b-2482-4dbc-bd84-fc5541bd934d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:00.602716 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8ea05b-2482-4dbc-bd84-fc5541bd934d None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:00.603000 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8ea05b-2482-4dbc-bd84-fc5541bd934d None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:00.603281 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8ea05b-2482-4dbc-bd84-fc5541bd934d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:00.604103 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8ea05b-2482-4dbc-bd84-fc5541bd934d None admin] RBAC: Adding query filter params (role.id=5ec7b68b793d4a54abd30460961346b9, scope.project.id=c95f8f3084c5472cb8364b7c57207394, user.id=16b49e866398400580ae48aa5a96fc04) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:00.604421 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8ea05b-2482-4dbc-bd84-fc5541bd934d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:00.605883 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8ea05b-2482-4dbc-bd84-fc5541bd934d None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:00.611178 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 286/581] 10.222.0.23 () {56 vars in 1374 bytes} [Sat Jun 18 04:59:00 2022] GET /identity/v3/role_assignments?user.id=16b49e866398400580ae48aa5a96fc04&scope.project.id=c95f8f3084c5472cb8364b7c57207394&role.id=5ec7b68b793d4a54abd30460961346b9 => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:00.611938 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8ea05b-2482-4dbc-bd84-fc5541bd934d None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:00.636172 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5e2ecd-c59e-4555-8ce2-06d0955c93ae None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:00.676123 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11be31e-0c61-4c49-a89e-b2e224f15805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:00.677033 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11be31e-0c61-4c49-a89e-b2e224f15805 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:00.677315 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11be31e-0c61-4c49-a89e-b2e224f15805 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:00.677684 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11be31e-0c61-4c49-a89e-b2e224f15805 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:00.690944 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11be31e-0c61-4c49-a89e-b2e224f15805 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:00.692586 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11be31e-0c61-4c49-a89e-b2e224f15805 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:00.712538 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-f11be31e-0c61-4c49-a89e-b2e224f15805 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 04:59:00.712538 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:00.712538 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:59:00.712538 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:59:00.712538 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:59:00.712538 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:59:00.712538 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:59:00.712538 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:59:00.712538 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:59:00.712538 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:59:00.712538 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:59:00.712538 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:59:00.712538 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 18 04:59:00.712538 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 18 04:59:00.712538 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 18 04:59:00.713629 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 18 04:59:00.713629 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:59:00.713629 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:59:00.713629 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:59:00.713629 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:59:00.713629 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:59:00.713629 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:59:00.713629 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:59:00.713629 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jun 18 04:59:00.713629 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:59:00.713629 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:59:00.713629 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:59:00.713629 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:59:00.713629 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:59:00.713629 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:59:00.713629 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:59:00.713629 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:59:00.714542 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 18 04:59:00.714542 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 18 04:59:00.714542 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 04:59:00.714542 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 04:59:00.714542 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 296/582] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Jun 18 04:59:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:59:00.714812 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11be31e-0c61-4c49-a89e-b2e224f15805 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:00.731931 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8ea05b-2482-4dbc-bd84-fc5541bd934d None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:00.781502 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:00.805821 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70035087-f9f5-4d28-9f05-941e248cc7b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:00.806622 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70035087-f9f5-4d28-9f05-941e248cc7b9 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:00.806903 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70035087-f9f5-4d28-9f05-941e248cc7b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:00.807175 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70035087-f9f5-4d28-9f05-941e248cc7b9 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:00.815089 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70035087-f9f5-4d28-9f05-941e248cc7b9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:00.816525 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70035087-f9f5-4d28-9f05-941e248cc7b9 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:00.831434 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application [None req-70035087-f9f5-4d28-9f05-941e248cc7b9 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 18 04:59:00.831434 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:00.831434 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:59:00.831434 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:59:00.831434 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:59:00.831434 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:59:00.831434 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:59:00.831434 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:59:00.831434 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:59:00.831434 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:59:00.831434 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:59:00.831434 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:59:00.831434 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 18 04:59:00.831434 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 18 04:59:00.831434 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 18 04:59:00.831434 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 18 04:59:00.832387 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:59:00.832387 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:59:00.832387 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 18 04:59:00.832387 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 18 04:59:00.832387 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 18 04:59:00.832387 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 18 04:59:00.832387 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:59:00.832387 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:59:00.832387 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:59:00.832387 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:59:00.832387 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:59:00.832387 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application with Lock( Jun 18 04:59:00.832387 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:59:00.832387 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:59:00.832387 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:59:00.832387 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:59:00.832387 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:59:00.832387 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:59:00.833357 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:59:00.833357 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:59:00.833357 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 18 04:59:00.833357 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 18 04:59:00.833357 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 18 04:59:00.833357 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 18 04:59:00.833357 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 18 04:59:00.833357 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 18 04:59:00.833357 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 18 04:59:00.833357 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 04:59:00.833357 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 287/583] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Jun 18 04:59:00 2022] GET /identity/v3/users/glance => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:59:00.833357 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70035087-f9f5-4d28-9f05-941e248cc7b9 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:00.839314 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11be31e-0c61-4c49-a89e-b2e224f15805 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:00.874610 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3a8c34-f92d-4f3d-ab57-5609178cac6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:00.875500 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3a8c34-f92d-4f3d-ab57-5609178cac6e None admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:00.875776 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3a8c34-f92d-4f3d-ab57-5609178cac6e None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:00.876055 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3a8c34-f92d-4f3d-ab57-5609178cac6e None admin] PATH_INFO: `/v3/projects/c95f8f3084c5472cb8364b7c57207394/users/15e7f45ee0744c1d9602187eac8e8b80/roles/5ec7b68b793d4a54abd30460961346b9` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:00.891761 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3a8c34-f92d-4f3d-ab57-5609178cac6e None admin] RBAC: Authorizing `identity:create_grant(project_id=c95f8f3084c5472cb8364b7c57207394, user_id=15e7f45ee0744c1d9602187eac8e8b80, role_id=5ec7b68b793d4a54abd30460961346b9)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:00.892265 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:00.893145 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3a8c34-f92d-4f3d-ab57-5609178cac6e None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:00.941083 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70035087-f9f5-4d28-9f05-941e248cc7b9 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:00.962076 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 297/584] 10.222.0.23 () {58 vars in 1439 bytes} [Sat Jun 18 04:59:00 2022] PUT /identity/v3/projects/c95f8f3084c5472cb8364b7c57207394/users/15e7f45ee0744c1d9602187eac8e8b80/roles/5ec7b68b793d4a54abd30460961346b9 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 04:59:00.963156 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3a8c34-f92d-4f3d-ab57-5609178cac6e None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:00.990805 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda421e5-e168-4ab8-93b8-08ea55532b20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:00.991666 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda421e5-e168-4ab8-93b8-08ea55532b20 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:00.991959 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda421e5-e168-4ab8-93b8-08ea55532b20 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:00.992241 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda421e5-e168-4ab8-93b8-08ea55532b20 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:01.009273 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda421e5-e168-4ab8-93b8-08ea55532b20 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:01.011154 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda421e5-e168-4ab8-93b8-08ea55532b20 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:01.031039 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application [None req-cda421e5-e168-4ab8-93b8-08ea55532b20 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 04:59:01.031039 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:01.031039 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:59:01.031039 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:59:01.031039 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:59:01.031039 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:59:01.031039 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:59:01.031039 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:59:01.031039 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:59:01.031039 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:59:01.031039 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:59:01.031039 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:59:01.031039 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 18 04:59:01.031039 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 18 04:59:01.031039 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 18 04:59:01.032065 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 18 04:59:01.032065 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:59:01.032065 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:59:01.032065 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:59:01.032065 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:59:01.032065 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:59:01.032065 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:59:01.032065 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:59:01.032065 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application with Lock( Jun 18 04:59:01.032065 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:59:01.032065 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:59:01.032065 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:59:01.032065 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:59:01.032065 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:59:01.032065 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:59:01.032065 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:59:01.032065 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:59:01.032953 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 18 04:59:01.032953 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 18 04:59:01.032953 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 04:59:01.032953 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 04:59:01.035065 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 288/585] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Jun 18 04:59:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:59:01.036227 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda421e5-e168-4ab8-93b8-08ea55532b20 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:01.105752 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:01.112257 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3a8c34-f92d-4f3d-ab57-5609178cac6e None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:01.161431 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda421e5-e168-4ab8-93b8-08ea55532b20 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:01.164669 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:01.167231 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7af02a-eae8-4fa1-a3f5-c0d1eaeb4c48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:01.167976 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7af02a-eae8-4fa1-a3f5-c0d1eaeb4c48 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:01.168326 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7af02a-eae8-4fa1-a3f5-c0d1eaeb4c48 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:01.168668 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7af02a-eae8-4fa1-a3f5-c0d1eaeb4c48 None admin] PATH_INFO: `/v3/domains` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:01.169497 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7af02a-eae8-4fa1-a3f5-c0d1eaeb4c48 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:01.170305 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7af02a-eae8-4fa1-a3f5-c0d1eaeb4c48 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:01.171758 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7af02a-eae8-4fa1-a3f5-c0d1eaeb4c48 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:01.188879 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 298/586] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Jun 18 04:59:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:01.189805 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7af02a-eae8-4fa1-a3f5-c0d1eaeb4c48 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:01.201639 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a2e6bc-609b-4933-9d55-414c987db083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:01.202481 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a2e6bc-609b-4933-9d55-414c987db083 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:01.203010 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a2e6bc-609b-4933-9d55-414c987db083 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:01.203371 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a2e6bc-609b-4933-9d55-414c987db083 None admin] PATH_INFO: `/v3/users` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:01.204897 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a2e6bc-609b-4933-9d55-414c987db083 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:01.205292 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a2e6bc-609b-4933-9d55-414c987db083 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:01.206952 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a2e6bc-609b-4933-9d55-414c987db083 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:01.229021 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 289/587] 10.222.0.23 () {56 vars in 1137 bytes} [Sat Jun 18 04:59:01 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:01.230180 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a2e6bc-609b-4933-9d55-414c987db083 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:01.306750 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7af02a-eae8-4fa1-a3f5-c0d1eaeb4c48 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:01.348732 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8539b19c-8662-4e03-a3bc-9b56a258ec55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:01.350076 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8539b19c-8662-4e03-a3bc-9b56a258ec55 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:01.350432 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8539b19c-8662-4e03-a3bc-9b56a258ec55 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:01.350547 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a2e6bc-609b-4933-9d55-414c987db083 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:01.350920 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8539b19c-8662-4e03-a3bc-9b56a258ec55 None admin] PATH_INFO: `/v3/domains` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:01.351721 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8539b19c-8662-4e03-a3bc-9b56a258ec55 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:01.352370 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8539b19c-8662-4e03-a3bc-9b56a258ec55 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:01.354860 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8539b19c-8662-4e03-a3bc-9b56a258ec55 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:01.371641 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 299/588] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Jun 18 04:59:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:01.372574 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8539b19c-8662-4e03-a3bc-9b56a258ec55 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:01.395628 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:01.400704 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d752cd2-860e-44c7-914f-8ee58f721dc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:01.401750 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d752cd2-860e-44c7-914f-8ee58f721dc6 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:01.402462 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d752cd2-860e-44c7-914f-8ee58f721dc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:01.408078 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d752cd2-860e-44c7-914f-8ee58f721dc6 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:01.411116 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d752cd2-860e-44c7-914f-8ee58f721dc6 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:01.413414 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d752cd2-860e-44c7-914f-8ee58f721dc6 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:01.429813 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application [None req-0d752cd2-860e-44c7-914f-8ee58f721dc6 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 18 04:59:01.429813 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:01.429813 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:59:01.429813 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:59:01.429813 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:59:01.429813 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:59:01.429813 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:59:01.429813 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:59:01.429813 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:59:01.429813 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:59:01.429813 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:59:01.429813 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:59:01.429813 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 18 04:59:01.429813 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 18 04:59:01.429813 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 18 04:59:01.429813 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 18 04:59:01.431367 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:59:01.431367 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:59:01.431367 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 18 04:59:01.431367 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 18 04:59:01.431367 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 18 04:59:01.431367 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 18 04:59:01.431367 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:59:01.431367 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:59:01.431367 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:59:01.431367 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:59:01.431367 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:59:01.431367 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application with Lock( Jun 18 04:59:01.431367 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:59:01.431367 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:59:01.431367 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:59:01.431367 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:59:01.431367 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:59:01.431367 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:59:01.432474 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:59:01.432474 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:59:01.432474 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 18 04:59:01.432474 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 18 04:59:01.432474 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 18 04:59:01.432474 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 18 04:59:01.432474 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 18 04:59:01.432474 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 18 04:59:01.432474 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 18 04:59:01.432474 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 04:59:01.434775 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 290/589] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Jun 18 04:59:01 2022] GET /identity/v3/users/cinder => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:59:01.435881 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d752cd2-860e-44c7-914f-8ee58f721dc6 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:01.464488 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:01.495420 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8539b19c-8662-4e03-a3bc-9b56a258ec55 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:01.535085 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caee5acb-68c2-4238-b94f-cd91b82f253a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:01.536136 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caee5acb-68c2-4238-b94f-cd91b82f253a None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:01.536567 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caee5acb-68c2-4238-b94f-cd91b82f253a None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:01.537159 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caee5acb-68c2-4238-b94f-cd91b82f253a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:01.543545 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d752cd2-860e-44c7-914f-8ee58f721dc6 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:01.548681 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caee5acb-68c2-4238-b94f-cd91b82f253a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:01.550439 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caee5acb-68c2-4238-b94f-cd91b82f253a None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:01.570796 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-caee5acb-68c2-4238-b94f-cd91b82f253a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 04:59:01.570796 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:01.570796 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:59:01.570796 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:59:01.570796 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:59:01.570796 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:59:01.570796 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:59:01.570796 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:59:01.570796 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:59:01.570796 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:59:01.570796 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:59:01.570796 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:59:01.570796 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 18 04:59:01.570796 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 18 04:59:01.570796 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 18 04:59:01.571971 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 18 04:59:01.571971 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:59:01.571971 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:59:01.571971 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:59:01.571971 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:59:01.571971 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:59:01.571971 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:59:01.571971 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:59:01.571971 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jun 18 04:59:01.571971 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:59:01.571971 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:59:01.571971 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:59:01.571971 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:59:01.571971 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:59:01.571971 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:59:01.571971 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:59:01.571971 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:59:01.573088 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 18 04:59:01.573088 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 18 04:59:01.573088 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 04:59:01.573088 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 04:59:01.575120 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 300/590] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Jun 18 04:59:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:59:01.576212 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caee5acb-68c2-4238-b94f-cd91b82f253a None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:01.586717 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee22d80-8505-452d-b3ee-75175861419d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:01.587659 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee22d80-8505-452d-b3ee-75175861419d None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:01.588045 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee22d80-8505-452d-b3ee-75175861419d None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:01.588419 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee22d80-8505-452d-b3ee-75175861419d None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:01.596090 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee22d80-8505-452d-b3ee-75175861419d None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:01.597671 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee22d80-8505-452d-b3ee-75175861419d None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:01.613088 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application [None req-2ee22d80-8505-452d-b3ee-75175861419d None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 18 04:59:01.613088 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:01.613088 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:59:01.613088 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:59:01.613088 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:59:01.613088 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:59:01.613088 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:59:01.613088 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:59:01.613088 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:59:01.613088 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:59:01.613088 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:59:01.613088 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:59:01.613088 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 18 04:59:01.613088 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 18 04:59:01.613088 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 18 04:59:01.613088 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 18 04:59:01.614243 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:59:01.614243 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:59:01.614243 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 18 04:59:01.614243 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 18 04:59:01.614243 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 18 04:59:01.614243 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 18 04:59:01.614243 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:59:01.614243 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:59:01.614243 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:59:01.614243 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:59:01.614243 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:59:01.614243 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application with Lock( Jun 18 04:59:01.614243 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:59:01.614243 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:59:01.614243 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:59:01.614243 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:59:01.614243 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:59:01.614243 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:59:01.615687 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:59:01.615687 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:59:01.615687 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 18 04:59:01.615687 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 18 04:59:01.615687 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 18 04:59:01.615687 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 18 04:59:01.615687 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 18 04:59:01.615687 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 18 04:59:01.615687 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 18 04:59:01.615687 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 04:59:01.621857 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 291/591] 10.222.0.23 () {56 vars in 1102 bytes} [Sat Jun 18 04:59:01 2022] GET /identity/v3/users/neutron => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:59:01.621857 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee22d80-8505-452d-b3ee-75175861419d None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:01.676085 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:01.716174 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caee5acb-68c2-4238-b94f-cd91b82f253a None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:01.718163 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:01.745868 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee22d80-8505-452d-b3ee-75175861419d None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:01.755782 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400ac9c6-5584-495a-a372-4fdbe81a8b07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:01.756997 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400ac9c6-5584-495a-a372-4fdbe81a8b07 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:01.757123 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400ac9c6-5584-495a-a372-4fdbe81a8b07 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:01.757538 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400ac9c6-5584-495a-a372-4fdbe81a8b07 None admin] PATH_INFO: `/v3/users` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:01.758484 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400ac9c6-5584-495a-a372-4fdbe81a8b07 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:01.758937 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400ac9c6-5584-495a-a372-4fdbe81a8b07 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:01.760533 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400ac9c6-5584-495a-a372-4fdbe81a8b07 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:01.780724 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 301/592] 10.222.0.23 () {56 vars in 1137 bytes} [Sat Jun 18 04:59:01 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:01.781830 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400ac9c6-5584-495a-a372-4fdbe81a8b07 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:01.794073 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4075ba4e-c1c2-4c1d-9fb5-a83bbcffd6f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:01.795273 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4075ba4e-c1c2-4c1d-9fb5-a83bbcffd6f6 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:01.796015 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4075ba4e-c1c2-4c1d-9fb5-a83bbcffd6f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:01.796444 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4075ba4e-c1c2-4c1d-9fb5-a83bbcffd6f6 None admin] PATH_INFO: `/v3/domains` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:01.797325 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4075ba4e-c1c2-4c1d-9fb5-a83bbcffd6f6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:01.797966 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4075ba4e-c1c2-4c1d-9fb5-a83bbcffd6f6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:01.800152 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4075ba4e-c1c2-4c1d-9fb5-a83bbcffd6f6 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:01.816404 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 292/593] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Jun 18 04:59:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:01.817628 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4075ba4e-c1c2-4c1d-9fb5-a83bbcffd6f6 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:01.908115 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400ac9c6-5584-495a-a372-4fdbe81a8b07 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:01.936153 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4075ba4e-c1c2-4c1d-9fb5-a83bbcffd6f6 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:01.943946 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:01.957922 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047d02b8-74f2-4423-8c25-dabe629ee98b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:01.958588 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047d02b8-74f2-4423-8c25-dabe629ee98b None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:01.958932 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047d02b8-74f2-4423-8c25-dabe629ee98b None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:01.959201 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047d02b8-74f2-4423-8c25-dabe629ee98b None admin] PATH_INFO: `/v3/users` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:01.960183 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047d02b8-74f2-4423-8c25-dabe629ee98b None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:01.960499 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047d02b8-74f2-4423-8c25-dabe629ee98b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:01.961972 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047d02b8-74f2-4423-8c25-dabe629ee98b None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:01.984936 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 302/594] 10.222.0.23 () {56 vars in 1139 bytes} [Sat Jun 18 04:59:01 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:01.985550 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8227568e-a7eb-4f65-8005-f59181590143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:01.985868 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047d02b8-74f2-4423-8c25-dabe629ee98b None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:01.986883 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8227568e-a7eb-4f65-8005-f59181590143 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:01.987365 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8227568e-a7eb-4f65-8005-f59181590143 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:01.987839 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8227568e-a7eb-4f65-8005-f59181590143 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:02.002937 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8227568e-a7eb-4f65-8005-f59181590143 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:02.004979 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8227568e-a7eb-4f65-8005-f59181590143 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:02.007998 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:02.028746 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application [None req-8227568e-a7eb-4f65-8005-f59181590143 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 04:59:02.028746 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:02.028746 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:59:02.028746 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:59:02.028746 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:59:02.028746 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:59:02.028746 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:59:02.028746 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:59:02.028746 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:59:02.028746 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:59:02.028746 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:59:02.028746 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:59:02.028746 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 18 04:59:02.028746 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 18 04:59:02.028746 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 18 04:59:02.029720 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 18 04:59:02.029720 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:59:02.029720 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:59:02.029720 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:59:02.029720 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:59:02.029720 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:59:02.029720 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:59:02.029720 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:59:02.029720 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application with Lock( Jun 18 04:59:02.029720 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:59:02.029720 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:59:02.029720 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:59:02.029720 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:59:02.029720 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:59:02.029720 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:59:02.029720 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:59:02.029720 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:59:02.030589 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 18 04:59:02.030589 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 18 04:59:02.030589 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 04:59:02.030589 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 04:59:02.033340 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 293/595] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Jun 18 04:59:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:59:02.034621 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8227568e-a7eb-4f65-8005-f59181590143 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:02.104699 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047d02b8-74f2-4423-8c25-dabe629ee98b None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:02.141950 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db3fe1b-a0f4-449c-b3d5-b58994e2a23b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:02.142903 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db3fe1b-a0f4-449c-b3d5-b58994e2a23b None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:02.143295 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db3fe1b-a0f4-449c-b3d5-b58994e2a23b None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:02.143784 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db3fe1b-a0f4-449c-b3d5-b58994e2a23b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:02.151139 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db3fe1b-a0f4-449c-b3d5-b58994e2a23b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:02.152709 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db3fe1b-a0f4-449c-b3d5-b58994e2a23b None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:02.159616 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8227568e-a7eb-4f65-8005-f59181590143 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:02.168808 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-9db3fe1b-a0f4-449c-b3d5-b58994e2a23b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 18 04:59:02.168808 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:02.168808 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:59:02.168808 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:59:02.168808 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:59:02.168808 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:59:02.168808 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:59:02.168808 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:59:02.168808 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:59:02.168808 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:59:02.168808 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:59:02.168808 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:59:02.168808 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 18 04:59:02.168808 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 18 04:59:02.168808 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 18 04:59:02.169932 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 18 04:59:02.169932 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:59:02.169932 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:59:02.169932 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:59:02.169932 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:59:02.169932 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:59:02.169932 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:59:02.169932 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:59:02.169932 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jun 18 04:59:02.169932 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:59:02.169932 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:59:02.169932 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:59:02.169932 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:59:02.169932 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:59:02.169932 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:59:02.169932 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:59:02.169932 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:59:02.170903 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 18 04:59:02.170903 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 18 04:59:02.170903 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 18 04:59:02.170903 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 18 04:59:02.170903 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 18 04:59:02.170903 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 18 04:59:02.170903 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 18 04:59:02.170903 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 04:59:02.176415 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 303/596] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Jun 18 04:59:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:59:02.177699 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db3fe1b-a0f4-449c-b3d5-b58994e2a23b None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:02.205482 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62eeedfc-3b47-4c6c-8849-2316c3645609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:02.207324 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62eeedfc-3b47-4c6c-8849-2316c3645609 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:02.208087 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62eeedfc-3b47-4c6c-8849-2316c3645609 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:02.209248 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62eeedfc-3b47-4c6c-8849-2316c3645609 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:02.223290 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62eeedfc-3b47-4c6c-8849-2316c3645609 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:02.225193 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62eeedfc-3b47-4c6c-8849-2316c3645609 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:02.242055 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:02.245267 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application [None req-62eeedfc-3b47-4c6c-8849-2316c3645609 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 04:59:02.245267 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:02.245267 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:59:02.245267 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:59:02.245267 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:59:02.245267 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:59:02.245267 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:59:02.245267 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:59:02.245267 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:59:02.245267 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:59:02.245267 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:59:02.245267 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:59:02.245267 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 18 04:59:02.245267 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 18 04:59:02.245267 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 18 04:59:02.246245 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 18 04:59:02.246245 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:59:02.246245 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:59:02.246245 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:59:02.246245 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:59:02.246245 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:59:02.246245 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:59:02.246245 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:59:02.246245 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application with Lock( Jun 18 04:59:02.246245 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:59:02.246245 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:59:02.246245 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:59:02.246245 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:59:02.246245 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:59:02.246245 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:59:02.246245 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:59:02.246245 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:59:02.247128 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 18 04:59:02.247128 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 18 04:59:02.247128 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 04:59:02.247128 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 04:59:02.251432 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 294/597] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Jun 18 04:59:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:59:02.252362 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62eeedfc-3b47-4c6c-8849-2316c3645609 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:02.304722 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:02.317090 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db3fe1b-a0f4-449c-b3d5-b58994e2a23b None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:02.356153 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577a4a28-975c-416a-ae19-3f531a037178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:02.356950 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577a4a28-975c-416a-ae19-3f531a037178 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:02.357463 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577a4a28-975c-416a-ae19-3f531a037178 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:02.357823 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577a4a28-975c-416a-ae19-3f531a037178 None admin] PATH_INFO: `/v3/domains` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:02.358727 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577a4a28-975c-416a-ae19-3f531a037178 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:02.359108 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577a4a28-975c-416a-ae19-3f531a037178 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:02.360572 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577a4a28-975c-416a-ae19-3f531a037178 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:02.374129 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62eeedfc-3b47-4c6c-8849-2316c3645609 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:02.375353 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 304/598] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Jun 18 04:59:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:02.376513 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577a4a28-975c-416a-ae19-3f531a037178 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:02.415370 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e366c00-bf47-4b75-ab34-7705cd0b8c98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:02.416360 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e366c00-bf47-4b75-ab34-7705cd0b8c98 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:02.416519 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e366c00-bf47-4b75-ab34-7705cd0b8c98 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:02.416905 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e366c00-bf47-4b75-ab34-7705cd0b8c98 None admin] PATH_INFO: `/v3/projects` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:02.417802 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e366c00-bf47-4b75-ab34-7705cd0b8c98 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:02.418228 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e366c00-bf47-4b75-ab34-7705cd0b8c98 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:02.419770 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e366c00-bf47-4b75-ab34-7705cd0b8c98 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:02.438114 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 295/599] 10.222.0.23 () {56 vars in 1148 bytes} [Sat Jun 18 04:59:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:02.439240 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e366c00-bf47-4b75-ab34-7705cd0b8c98 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:02.500446 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577a4a28-975c-416a-ae19-3f531a037178 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:02.510677 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:02.543246 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e55534e-508c-4400-973c-248c13bee5f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:02.544229 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e55534e-508c-4400-973c-248c13bee5f9 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:02.544518 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e55534e-508c-4400-973c-248c13bee5f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:02.544796 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e55534e-508c-4400-973c-248c13bee5f9 None admin] PATH_INFO: `/v3/domains` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:02.545847 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e55534e-508c-4400-973c-248c13bee5f9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:02.546230 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e55534e-508c-4400-973c-248c13bee5f9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:02.547850 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e55534e-508c-4400-973c-248c13bee5f9 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:02.553701 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e366c00-bf47-4b75-ab34-7705cd0b8c98 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:02.562243 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:02.569327 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 305/600] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Jun 18 04:59:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:02.569327 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e55534e-508c-4400-973c-248c13bee5f9 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:02.598015 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d40cee9-c6cf-4153-bae7-f35bd852d65a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:02.599136 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d40cee9-c6cf-4153-bae7-f35bd852d65a None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:02.599885 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d40cee9-c6cf-4153-bae7-f35bd852d65a None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:02.600388 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d40cee9-c6cf-4153-bae7-f35bd852d65a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:02.607873 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d40cee9-c6cf-4153-bae7-f35bd852d65a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:02.609676 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d40cee9-c6cf-4153-bae7-f35bd852d65a None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:02.624461 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application [None req-0d40cee9-c6cf-4153-bae7-f35bd852d65a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 18 04:59:02.624461 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:02.624461 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:59:02.624461 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:59:02.624461 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:59:02.624461 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:59:02.624461 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:59:02.624461 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:59:02.624461 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:59:02.624461 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:59:02.624461 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:59:02.624461 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:59:02.624461 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 18 04:59:02.624461 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 18 04:59:02.624461 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 18 04:59:02.625671 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 18 04:59:02.625671 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:59:02.625671 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:59:02.625671 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:59:02.625671 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:59:02.625671 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:59:02.625671 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:59:02.625671 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:59:02.625671 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application with Lock( Jun 18 04:59:02.625671 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:59:02.625671 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:59:02.625671 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:59:02.625671 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:59:02.625671 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:59:02.625671 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:59:02.625671 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:59:02.625671 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:59:02.626599 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 18 04:59:02.626599 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 18 04:59:02.626599 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 18 04:59:02.626599 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 18 04:59:02.626599 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 18 04:59:02.626599 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 18 04:59:02.626599 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 18 04:59:02.626599 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 04:59:02.628942 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 296/601] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Jun 18 04:59:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:59:02.630030 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d40cee9-c6cf-4153-bae7-f35bd852d65a None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:02.688985 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e55534e-508c-4400-973c-248c13bee5f9 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:02.730967 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0c29b7-b029-482a-a8f0-d62ae43c77db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:02.731851 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0c29b7-b029-482a-a8f0-d62ae43c77db None admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:02.732127 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0c29b7-b029-482a-a8f0-d62ae43c77db None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:02.732396 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0c29b7-b029-482a-a8f0-d62ae43c77db None admin] PATH_INFO: `/v3/projects/c95f8f3084c5472cb8364b7c57207394/users/4638e5d1fa124667b07054a87c2d353e/roles/5ec7b68b793d4a54abd30460961346b9` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:02.748084 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0c29b7-b029-482a-a8f0-d62ae43c77db None admin] RBAC: Authorizing `identity:create_grant(project_id=c95f8f3084c5472cb8364b7c57207394, user_id=4638e5d1fa124667b07054a87c2d353e, role_id=5ec7b68b793d4a54abd30460961346b9)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:02.749680 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0c29b7-b029-482a-a8f0-d62ae43c77db None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:02.750597 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d40cee9-c6cf-4153-bae7-f35bd852d65a None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:02.786444 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baabe048-c47e-474d-8903-e42d29f3c5db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:02.787504 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baabe048-c47e-474d-8903-e42d29f3c5db None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:02.787786 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baabe048-c47e-474d-8903-e42d29f3c5db None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:02.788061 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baabe048-c47e-474d-8903-e42d29f3c5db None admin] PATH_INFO: `/v3/projects/service` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:02.794663 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baabe048-c47e-474d-8903-e42d29f3c5db None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:02.796202 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baabe048-c47e-474d-8903-e42d29f3c5db None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:02.810229 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application [None req-baabe048-c47e-474d-8903-e42d29f3c5db None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 18 04:59:02.810229 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:02.810229 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:59:02.810229 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:59:02.810229 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:59:02.810229 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:59:02.810229 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:59:02.810229 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:59:02.810229 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:59:02.810229 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:59:02.810229 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:59:02.810229 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:59:02.810229 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 18 04:59:02.810229 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 18 04:59:02.810229 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 18 04:59:02.811119 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 18 04:59:02.811119 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:59:02.811119 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:59:02.811119 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:59:02.811119 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:59:02.811119 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:59:02.811119 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:59:02.811119 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:59:02.811119 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application with Lock( Jun 18 04:59:02.811119 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:59:02.811119 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:59:02.811119 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:59:02.811119 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:59:02.811119 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:59:02.811119 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:59:02.811119 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:59:02.811119 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:59:02.811969 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 18 04:59:02.811969 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 18 04:59:02.811969 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 18 04:59:02.811969 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 18 04:59:02.811969 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 18 04:59:02.811969 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 18 04:59:02.811969 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 18 04:59:02.811969 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 04:59:02.811969 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 297/602] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Jun 18 04:59:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:59:02.812439 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baabe048-c47e-474d-8903-e42d29f3c5db None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:02.831483 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 306/603] 10.222.0.23 () {58 vars in 1439 bytes} [Sat Jun 18 04:59:02 2022] PUT /identity/v3/projects/c95f8f3084c5472cb8364b7c57207394/users/4638e5d1fa124667b07054a87c2d353e/roles/5ec7b68b793d4a54abd30460961346b9 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 04:59:02.832889 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1857401d-6fc3-4468-8b17-5de90dca9190 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:02.833183 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1857401d-6fc3-4468-8b17-5de90dca9190 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:02.833586 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1857401d-6fc3-4468-8b17-5de90dca9190 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:02.839527 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 307/604] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 04:59:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 04:59:02.839527 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1857401d-6fc3-4468-8b17-5de90dca9190 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:02.860230 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:02.863603 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:02.964916 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baabe048-c47e-474d-8903-e42d29f3c5db None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:02.977695 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1857401d-6fc3-4468-8b17-5de90dca9190 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:03.009039 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46e0120-018a-4c46-8dcb-038f4bb9b2e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:03.009818 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46e0120-018a-4c46-8dcb-038f4bb9b2e8 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:03.010105 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46e0120-018a-4c46-8dcb-038f4bb9b2e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:03.010380 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46e0120-018a-4c46-8dcb-038f4bb9b2e8 None admin] PATH_INFO: `/v3/projects` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:03.011665 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46e0120-018a-4c46-8dcb-038f4bb9b2e8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:03.011665 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46e0120-018a-4c46-8dcb-038f4bb9b2e8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:03.012912 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46e0120-018a-4c46-8dcb-038f4bb9b2e8 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:03.025750 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 298/605] 10.222.0.23 () {56 vars in 1148 bytes} [Sat Jun 18 04:59:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:03.026814 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6e638c-da46-477c-bed8-86c3683a38c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:03.026947 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96f9841-cdc3-4d8d-8e30-d9f67e3c0c12 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:03.027141 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96f9841-cdc3-4d8d-8e30-d9f67e3c0c12 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:03.027372 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96f9841-cdc3-4d8d-8e30-d9f67e3c0c12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:03.027534 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6e638c-da46-477c-bed8-86c3683a38c0 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:03.027804 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6e638c-da46-477c-bed8-86c3683a38c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:03.028175 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6e638c-da46-477c-bed8-86c3683a38c0 None admin] PATH_INFO: `/v3/projects` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:03.029107 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6e638c-da46-477c-bed8-86c3683a38c0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:03.029370 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6e638c-da46-477c-bed8-86c3683a38c0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:03.030879 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6e638c-da46-477c-bed8-86c3683a38c0 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:03.044893 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 308/606] 10.222.0.23 () {56 vars in 1148 bytes} [Sat Jun 18 04:59:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:03.045653 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6e638c-da46-477c-bed8-86c3683a38c0 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:03.050840 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-a96f9841-cdc3-4d8d-8e30-d9f67e3c0c12 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 04:59:03.051390 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-a96f9841-cdc3-4d8d-8e30-d9f67e3c0c12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 04:59:03.136907 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 299/607] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 04:59:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 18 04:59:03.137718 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:03.137833 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96f9841-cdc3-4d8d-8e30-d9f67e3c0c12 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:03.159333 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:03.173881 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6e638c-da46-477c-bed8-86c3683a38c0 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:03.210723 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647321c6-89b9-4e86-9088-90a74d264e36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:03.211614 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647321c6-89b9-4e86-9088-90a74d264e36 None admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:03.211894 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647321c6-89b9-4e86-9088-90a74d264e36 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:03.212161 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647321c6-89b9-4e86-9088-90a74d264e36 None admin] PATH_INFO: `/v3/projects/c95f8f3084c5472cb8364b7c57207394/users/d9de0428abc8400aa338aba4f7c34c94/roles/5ec7b68b793d4a54abd30460961346b9` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:03.228276 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647321c6-89b9-4e86-9088-90a74d264e36 None admin] RBAC: Authorizing `identity:create_grant(project_id=c95f8f3084c5472cb8364b7c57207394, user_id=d9de0428abc8400aa338aba4f7c34c94, role_id=5ec7b68b793d4a54abd30460961346b9)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:03.229769 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647321c6-89b9-4e86-9088-90a74d264e36 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:03.251339 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96f9841-cdc3-4d8d-8e30-d9f67e3c0c12 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:03.286301 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6fa592-3016-4afa-a6a5-2757a9777505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:03.288225 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6fa592-3016-4afa-a6a5-2757a9777505 None admin] REQUEST_METHOD: `PUT` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:03.288510 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6fa592-3016-4afa-a6a5-2757a9777505 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:03.288785 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6fa592-3016-4afa-a6a5-2757a9777505 None admin] PATH_INFO: `/v3/projects/c95f8f3084c5472cb8364b7c57207394/users/e78c4d7cde8f4ad9aa479bb09e325db3/roles/5ec7b68b793d4a54abd30460961346b9` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:03.303343 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6fa592-3016-4afa-a6a5-2757a9777505 None admin] RBAC: Authorizing `identity:create_grant(project_id=c95f8f3084c5472cb8364b7c57207394, user_id=e78c4d7cde8f4ad9aa479bb09e325db3, role_id=5ec7b68b793d4a54abd30460961346b9)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:03.304825 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6fa592-3016-4afa-a6a5-2757a9777505 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:03.319608 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 309/608] 10.222.0.23 () {58 vars in 1439 bytes} [Sat Jun 18 04:59:03 2022] PUT /identity/v3/projects/c95f8f3084c5472cb8364b7c57207394/users/d9de0428abc8400aa338aba4f7c34c94/roles/5ec7b68b793d4a54abd30460961346b9 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 04:59:03.321517 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412e0730-ca13-4b17-ba9a-9a5d061344a9 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:03.321517 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412e0730-ca13-4b17-ba9a-9a5d061344a9 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:03.321672 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412e0730-ca13-4b17-ba9a-9a5d061344a9 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:03.325482 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 310/609] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 04:59:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 04:59:03.325482 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412e0730-ca13-4b17-ba9a-9a5d061344a9 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:03.376114 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 300/610] 10.222.0.23 () {58 vars in 1439 bytes} [Sat Jun 18 04:59:03 2022] PUT /identity/v3/projects/c95f8f3084c5472cb8364b7c57207394/users/e78c4d7cde8f4ad9aa479bb09e325db3/roles/5ec7b68b793d4a54abd30460961346b9 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 04:59:03.377803 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcc71e7-6b01-40e1-8f1c-01b12a9344dd None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:03.378271 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcc71e7-6b01-40e1-8f1c-01b12a9344dd None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:03.378770 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcc71e7-6b01-40e1-8f1c-01b12a9344dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:03.423859 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-8fcc71e7-6b01-40e1-8f1c-01b12a9344dd None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 04:59:03.426705 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-8fcc71e7-6b01-40e1-8f1c-01b12a9344dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 04:59:03.549837 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412e0730-ca13-4b17-ba9a-9a5d061344a9 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:03.554837 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:03.567537 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 301/611] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 04:59:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 18 04:59:03.586738 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:03.594782 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a747fe-0d28-43ce-9d69-b292fb64c190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:03.595672 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a747fe-0d28-43ce-9d69-b292fb64c190 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:03.596015 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a747fe-0d28-43ce-9d69-b292fb64c190 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:03.596475 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a747fe-0d28-43ce-9d69-b292fb64c190 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:03.596759 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcc71e7-6b01-40e1-8f1c-01b12a9344dd None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:03.612215 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a747fe-0d28-43ce-9d69-b292fb64c190 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 18 04:59:03.620297 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-52a747fe-0d28-43ce-9d69-b292fb64c190 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 18 04:59:03.620297 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:03.620297 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:59:03.620297 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:59:03.620297 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:59:03.620297 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:59:03.620297 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:59:03.620297 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:59:03.620297 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:59:03.620297 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:59:03.620297 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:59:03.620297 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:59:03.620297 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 18 04:59:03.620297 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 18 04:59:03.620297 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 18 04:59:03.620297 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 18 04:59:03.629867 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 18 04:59:03.629867 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 18 04:59:03.629867 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 18 04:59:03.629867 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 18 04:59:03.629867 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:59:03.629867 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:59:03.629867 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:59:03.629867 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:59:03.629867 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:59:03.629867 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:59:03.629867 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:59:03.629867 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jun 18 04:59:03.629867 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:59:03.629867 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:59:03.629867 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:59:03.629867 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:59:03.631298 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:59:03.631298 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:59:03.631298 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:59:03.631298 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:59:03.631298 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 18 04:59:03.631298 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 18 04:59:03.631298 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 18 04:59:03.631298 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 18 04:59:03.631298 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 18 04:59:03.631298 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 18 04:59:03.631298 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 18 04:59:03.631298 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 04:59:03.631298 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 311/612] 10.222.0.23 () {56 vars in 1102 bytes} [Sat Jun 18 04:59:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:59:03.631298 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a747fe-0d28-43ce-9d69-b292fb64c190 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:03.764213 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a747fe-0d28-43ce-9d69-b292fb64c190 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:03.800010 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4296b571-e6df-4c6f-be8d-fa2b09794e97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:03.800911 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4296b571-e6df-4c6f-be8d-fa2b09794e97 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:03.801345 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4296b571-e6df-4c6f-be8d-fa2b09794e97 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:03.801797 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4296b571-e6df-4c6f-be8d-fa2b09794e97 None admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:03.803640 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4296b571-e6df-4c6f-be8d-fa2b09794e97 None admin] RBAC: Adding query filter params (name=service) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:03.804139 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4296b571-e6df-4c6f-be8d-fa2b09794e97 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:03.805708 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4296b571-e6df-4c6f-be8d-fa2b09794e97 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:03.809411 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcc71e7-6b01-40e1-8f1c-01b12a9344dd None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:03.815150 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 312/613] 10.222.0.23 () {56 vars in 1103 bytes} [Sat Jun 18 04:59:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 18 04:59:03.820894 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4296b571-e6df-4c6f-be8d-fa2b09794e97 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:03.877573 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413270ce-e2bc-47b9-9223-c7c5d65da03b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:03.879065 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413270ce-e2bc-47b9-9223-c7c5d65da03b None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:03.880986 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413270ce-e2bc-47b9-9223-c7c5d65da03b None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:03.881716 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413270ce-e2bc-47b9-9223-c7c5d65da03b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:03.906668 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413270ce-e2bc-47b9-9223-c7c5d65da03b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 18 04:59:03.926201 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application [None req-413270ce-e2bc-47b9-9223-c7c5d65da03b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 18 04:59:03.926201 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:03.926201 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:59:03.926201 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:59:03.926201 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:59:03.926201 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:59:03.926201 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:59:03.926201 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:59:03.926201 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:59:03.926201 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:59:03.926201 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:59:03.926201 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:59:03.926201 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 18 04:59:03.926201 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 18 04:59:03.926201 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 18 04:59:03.926201 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 18 04:59:03.927378 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 18 04:59:03.927378 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 18 04:59:03.927378 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 18 04:59:03.927378 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 18 04:59:03.927378 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:59:03.927378 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:59:03.927378 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:59:03.927378 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:59:03.927378 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:59:03.927378 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:59:03.927378 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:59:03.927378 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application with Lock( Jun 18 04:59:03.927378 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:59:03.927378 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:59:03.927378 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:59:03.927378 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:59:03.928810 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:59:03.928810 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:59:03.928810 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:59:03.928810 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:59:03.928810 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 18 04:59:03.928810 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 18 04:59:03.928810 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 18 04:59:03.928810 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 18 04:59:03.928810 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 18 04:59:03.928810 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 18 04:59:03.928810 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 18 04:59:03.928810 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 04:59:03.932903 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 302/614] 10.222.0.23 () {56 vars in 1102 bytes} [Sat Jun 18 04:59:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 337 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 18 04:59:03.937433 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:03.950767 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413270ce-e2bc-47b9-9223-c7c5d65da03b None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:04.025786 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:04.063690 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4296b571-e6df-4c6f-be8d-fa2b09794e97 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:04.067123 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413270ce-e2bc-47b9-9223-c7c5d65da03b None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:04.133864 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742f9d88-72f9-4869-813a-30967a666d43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:04.134534 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742f9d88-72f9-4869-813a-30967a666d43 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:04.134817 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742f9d88-72f9-4869-813a-30967a666d43 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:04.135089 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742f9d88-72f9-4869-813a-30967a666d43 None admin] PATH_INFO: `/v3/roles` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:04.135841 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742f9d88-72f9-4869-813a-30967a666d43 None admin] RBAC: Adding query filter params (name=service) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:04.136147 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742f9d88-72f9-4869-813a-30967a666d43 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:04.137533 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742f9d88-72f9-4869-813a-30967a666d43 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:04.141303 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2831ea99-1d69-4fe9-b0f1-cfdc21e8f0ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:04.142111 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2831ea99-1d69-4fe9-b0f1-cfdc21e8f0ba None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:04.142388 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2831ea99-1d69-4fe9-b0f1-cfdc21e8f0ba None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:04.142659 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2831ea99-1d69-4fe9-b0f1-cfdc21e8f0ba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:04.159146 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 303/615] 10.222.0.23 () {56 vars in 1103 bytes} [Sat Jun 18 04:59:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 18 04:59:04.161036 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2831ea99-1d69-4fe9-b0f1-cfdc21e8f0ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:04.163551 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742f9d88-72f9-4869-813a-30967a666d43 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:04.164750 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2831ea99-1d69-4fe9-b0f1-cfdc21e8f0ba None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:04.194844 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-2831ea99-1d69-4fe9-b0f1-cfdc21e8f0ba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 04:59:04.194844 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:04.194844 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:59:04.194844 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:59:04.194844 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:59:04.194844 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:59:04.194844 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:59:04.194844 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:59:04.194844 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:59:04.194844 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:59:04.194844 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:59:04.194844 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:59:04.194844 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 18 04:59:04.194844 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 18 04:59:04.194844 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 18 04:59:04.195805 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 18 04:59:04.195805 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:59:04.195805 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:59:04.195805 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:59:04.195805 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:59:04.195805 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:59:04.195805 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:59:04.195805 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:59:04.195805 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jun 18 04:59:04.195805 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:59:04.195805 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:59:04.195805 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:59:04.195805 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:59:04.195805 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:59:04.195805 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:59:04.195805 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:59:04.195805 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:59:04.196682 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 18 04:59:04.196682 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 18 04:59:04.196682 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 04:59:04.196682 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 04:59:04.196682 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 313/616] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Jun 18 04:59:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 376 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:59:04.202549 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2831ea99-1d69-4fe9-b0f1-cfdc21e8f0ba None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:04.364847 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742f9d88-72f9-4869-813a-30967a666d43 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:04.405731 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:04.415398 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:04.421208 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2831ea99-1d69-4fe9-b0f1-cfdc21e8f0ba None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:04.437307 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eabead9-1993-4e68-8d9e-dc80598874e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:04.438111 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eabead9-1993-4e68-8d9e-dc80598874e1 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:04.438398 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eabead9-1993-4e68-8d9e-dc80598874e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:04.438669 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eabead9-1993-4e68-8d9e-dc80598874e1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:04.460141 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eabead9-1993-4e68-8d9e-dc80598874e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:04.462625 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eabead9-1993-4e68-8d9e-dc80598874e1 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:04.487212 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application [None req-1eabead9-1993-4e68-8d9e-dc80598874e1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 04:59:04.487212 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:04.487212 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:59:04.487212 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:59:04.487212 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:59:04.487212 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:59:04.487212 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:59:04.487212 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:59:04.487212 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:59:04.487212 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:59:04.487212 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:59:04.487212 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:59:04.487212 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 18 04:59:04.487212 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 18 04:59:04.487212 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 18 04:59:04.488129 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 18 04:59:04.488129 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:59:04.488129 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:59:04.488129 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:59:04.488129 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:59:04.488129 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:59:04.488129 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:59:04.488129 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:59:04.488129 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application with Lock( Jun 18 04:59:04.488129 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:59:04.488129 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:59:04.488129 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:59:04.488129 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:59:04.488129 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:59:04.488129 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:59:04.488129 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:59:04.488129 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:59:04.488988 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 18 04:59:04.488988 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 18 04:59:04.488988 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 04:59:04.488988 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 04:59:04.489699 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 304/617] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Jun 18 04:59:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:59:04.494894 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec8e5e6-d361-4fd5-aabe-dc1e9d7d6a4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:04.495564 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec8e5e6-d361-4fd5-aabe-dc1e9d7d6a4b None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:04.495842 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec8e5e6-d361-4fd5-aabe-dc1e9d7d6a4b None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:04.496119 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec8e5e6-d361-4fd5-aabe-dc1e9d7d6a4b None admin] PATH_INFO: `/v3/domains` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:04.497238 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec8e5e6-d361-4fd5-aabe-dc1e9d7d6a4b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:04.497583 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec8e5e6-d361-4fd5-aabe-dc1e9d7d6a4b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:04.498948 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec8e5e6-d361-4fd5-aabe-dc1e9d7d6a4b None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:04.501195 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eabead9-1993-4e68-8d9e-dc80598874e1 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:04.524049 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 314/618] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Jun 18 04:59:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:04.529038 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec8e5e6-d361-4fd5-aabe-dc1e9d7d6a4b None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:04.631801 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eabead9-1993-4e68-8d9e-dc80598874e1 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:04.670703 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8a8ade-bd42-41af-bcfd-c9a36bcc362d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:04.671556 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8a8ade-bd42-41af-bcfd-c9a36bcc362d None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:04.671817 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8a8ade-bd42-41af-bcfd-c9a36bcc362d None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:04.672102 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8a8ade-bd42-41af-bcfd-c9a36bcc362d None admin] PATH_INFO: `/v3/domains` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:04.672845 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8a8ade-bd42-41af-bcfd-c9a36bcc362d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:04.673163 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8a8ade-bd42-41af-bcfd-c9a36bcc362d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:04.674771 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8a8ade-bd42-41af-bcfd-c9a36bcc362d None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:04.689751 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 305/619] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Jun 18 04:59:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:04.695978 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8a8ade-bd42-41af-bcfd-c9a36bcc362d None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:04.730457 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:04.774623 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec8e5e6-d361-4fd5-aabe-dc1e9d7d6a4b None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:04.822621 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5979cf2c-3f4d-42fe-98de-dd6b3d48f6c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:04.823416 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5979cf2c-3f4d-42fe-98de-dd6b3d48f6c6 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:04.823693 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5979cf2c-3f4d-42fe-98de-dd6b3d48f6c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:04.823966 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5979cf2c-3f4d-42fe-98de-dd6b3d48f6c6 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:04.837206 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5979cf2c-3f4d-42fe-98de-dd6b3d48f6c6 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:04.842127 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5979cf2c-3f4d-42fe-98de-dd6b3d48f6c6 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:04.862036 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-5979cf2c-3f4d-42fe-98de-dd6b3d48f6c6 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 18 04:59:04.862036 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:04.862036 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:59:04.862036 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:59:04.862036 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:59:04.862036 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:59:04.862036 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:59:04.862036 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:59:04.862036 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:59:04.862036 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:59:04.862036 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:59:04.862036 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:59:04.862036 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 18 04:59:04.862036 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 18 04:59:04.862036 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 18 04:59:04.862036 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 18 04:59:04.863064 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:59:04.863064 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:59:04.863064 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 18 04:59:04.863064 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 18 04:59:04.863064 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 18 04:59:04.863064 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 18 04:59:04.863064 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:59:04.863064 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:59:04.863064 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:59:04.863064 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:59:04.863064 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:59:04.863064 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jun 18 04:59:04.863064 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:59:04.863064 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:59:04.863064 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:59:04.863064 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:59:04.863064 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:59:04.863064 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:59:04.864023 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:59:04.864023 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:59:04.864023 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 18 04:59:04.864023 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 18 04:59:04.864023 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 18 04:59:04.864023 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 18 04:59:04.864023 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 18 04:59:04.864023 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 18 04:59:04.864023 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 18 04:59:04.864023 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 04:59:04.868879 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 315/620] 10.222.0.23 () {56 vars in 1096 bytes} [Sat Jun 18 04:59:04 2022] GET /identity/v3/users/swift => generated 83 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 18 04:59:04.870086 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5979cf2c-3f4d-42fe-98de-dd6b3d48f6c6 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:04.872831 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8a8ade-bd42-41af-bcfd-c9a36bcc362d None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:04.909804 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:04.950294 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc7cda6-4ce3-4d18-b384-19dd3c58e104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:04.951090 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc7cda6-4ce3-4d18-b384-19dd3c58e104 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:04.951374 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc7cda6-4ce3-4d18-b384-19dd3c58e104 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:04.951650 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc7cda6-4ce3-4d18-b384-19dd3c58e104 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:04.966196 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc7cda6-4ce3-4d18-b384-19dd3c58e104 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:04.967929 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc7cda6-4ce3-4d18-b384-19dd3c58e104 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:04.985355 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application [None req-8dc7cda6-4ce3-4d18-b384-19dd3c58e104 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 18 04:59:04.985355 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:04.985355 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:59:04.985355 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:59:04.985355 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:59:04.985355 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:59:04.985355 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:59:04.985355 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:59:04.985355 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:59:04.985355 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:59:04.985355 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:59:04.985355 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:59:04.985355 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 18 04:59:04.985355 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 18 04:59:04.985355 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 18 04:59:04.985355 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 18 04:59:04.986358 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:59:04.986358 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:59:04.986358 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 18 04:59:04.986358 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 18 04:59:04.986358 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 18 04:59:04.986358 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 18 04:59:04.986358 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:59:04.986358 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:59:04.986358 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:59:04.986358 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:59:04.986358 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:59:04.986358 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application with Lock( Jun 18 04:59:04.986358 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:59:04.986358 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:59:04.986358 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:59:04.986358 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:59:04.986358 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:59:04.986358 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:59:04.987293 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:59:04.987293 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:59:04.987293 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 18 04:59:04.987293 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 18 04:59:04.987293 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 18 04:59:04.987293 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 18 04:59:04.987293 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 18 04:59:04.987293 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 18 04:59:04.987293 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 18 04:59:04.987293 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 04:59:04.987293 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 306/621] 10.222.0.23 () {56 vars in 1093 bytes} [Sat Jun 18 04:59:04 2022] GET /identity/v3/users/nova => generated 82 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 18 04:59:04.992496 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc7cda6-4ce3-4d18-b384-19dd3c58e104 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:05.055330 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5979cf2c-3f4d-42fe-98de-dd6b3d48f6c6 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:05.090558 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa120df-5c01-48b2-ac50-fc804ce6dc45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:05.091225 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa120df-5c01-48b2-ac50-fc804ce6dc45 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:05.091504 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa120df-5c01-48b2-ac50-fc804ce6dc45 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:05.091774 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa120df-5c01-48b2-ac50-fc804ce6dc45 None admin] PATH_INFO: `/v3/users` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:05.092599 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa120df-5c01-48b2-ac50-fc804ce6dc45 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:05.092909 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa120df-5c01-48b2-ac50-fc804ce6dc45 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:05.097479 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa120df-5c01-48b2-ac50-fc804ce6dc45 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:05.117257 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 316/622] 10.222.0.23 () {56 vars in 1135 bytes} [Sat Jun 18 04:59:04 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 18 04:59:05.118852 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d1510e-b819-4de3-90d5-ea815134aa65 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:05.119136 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d1510e-b819-4de3-90d5-ea815134aa65 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:05.119409 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d1510e-b819-4de3-90d5-ea815134aa65 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:05.120800 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 317/623] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 04:59:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 04:59:05.121936 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d1510e-b819-4de3-90d5-ea815134aa65 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:05.147138 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:05.156681 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc7cda6-4ce3-4d18-b384-19dd3c58e104 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:05.196289 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3521b11b-b3f6-4dcd-87c1-0f1180d0e618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:05.197212 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3521b11b-b3f6-4dcd-87c1-0f1180d0e618 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:05.198168 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3521b11b-b3f6-4dcd-87c1-0f1180d0e618 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:05.198460 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3521b11b-b3f6-4dcd-87c1-0f1180d0e618 None admin] PATH_INFO: `/v3/users` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:05.199591 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3521b11b-b3f6-4dcd-87c1-0f1180d0e618 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:05.199912 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3521b11b-b3f6-4dcd-87c1-0f1180d0e618 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:05.202604 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3521b11b-b3f6-4dcd-87c1-0f1180d0e618 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:05.212144 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:05.226877 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 307/624] 10.222.0.23 () {56 vars in 1133 bytes} [Sat Jun 18 04:59:04 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:05.228836 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975f4411-5545-49ce-a053-839afa174ef8 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:05.229598 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975f4411-5545-49ce-a053-839afa174ef8 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:05.230061 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975f4411-5545-49ce-a053-839afa174ef8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:05.236636 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d1510e-b819-4de3-90d5-ea815134aa65 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:05.253635 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-975f4411-5545-49ce-a053-839afa174ef8 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 04:59:05.255347 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-975f4411-5545-49ce-a053-839afa174ef8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 04:59:05.281238 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a0db8c-ffb1-41bb-b556-90d999384b1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:05.282303 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a0db8c-ffb1-41bb-b556-90d999384b1a None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:05.282648 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a0db8c-ffb1-41bb-b556-90d999384b1a None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:05.282986 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a0db8c-ffb1-41bb-b556-90d999384b1a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:05.294644 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a0db8c-ffb1-41bb-b556-90d999384b1a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:05.296276 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a0db8c-ffb1-41bb-b556-90d999384b1a None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:05.316185 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-12a0db8c-ffb1-41bb-b556-90d999384b1a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 04:59:05.316185 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:05.316185 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:59:05.316185 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:59:05.316185 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:59:05.316185 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:59:05.316185 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:59:05.316185 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:59:05.316185 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:59:05.316185 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:59:05.316185 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:59:05.316185 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:59:05.316185 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 18 04:59:05.316185 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 18 04:59:05.316185 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 18 04:59:05.317326 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 18 04:59:05.317326 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:59:05.317326 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:59:05.317326 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:59:05.317326 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:59:05.317326 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:59:05.317326 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:59:05.317326 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:59:05.317326 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jun 18 04:59:05.317326 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:59:05.317326 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:59:05.317326 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:59:05.317326 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:59:05.317326 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:59:05.317326 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:59:05.317326 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:59:05.317326 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:59:05.319298 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 18 04:59:05.319298 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 18 04:59:05.319298 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 04:59:05.319298 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 04:59:05.325466 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 318/625] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Jun 18 04:59:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 18 04:59:05.326521 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a0db8c-ffb1-41bb-b556-90d999384b1a None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:05.349996 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 308/626] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 04:59:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 18 04:59:05.349996 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975f4411-5545-49ce-a053-839afa174ef8 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:05.464110 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a0db8c-ffb1-41bb-b556-90d999384b1a None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:05.466894 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:05.496554 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975f4411-5545-49ce-a053-839afa174ef8 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:05.498253 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:05.505425 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b35f2b3-6bd7-4bd8-864a-7cbc46fab190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:05.507160 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b35f2b3-6bd7-4bd8-864a-7cbc46fab190 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:05.507655 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b35f2b3-6bd7-4bd8-864a-7cbc46fab190 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:05.510510 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b35f2b3-6bd7-4bd8-864a-7cbc46fab190 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:05.545673 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e129a3d-7c1a-400b-bf61-dfa92264c23d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:05.546351 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e129a3d-7c1a-400b-bf61-dfa92264c23d None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:05.546638 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e129a3d-7c1a-400b-bf61-dfa92264c23d None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:05.546913 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e129a3d-7c1a-400b-bf61-dfa92264c23d None admin] PATH_INFO: `/v3/domains` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:05.547899 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e129a3d-7c1a-400b-bf61-dfa92264c23d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:05.548214 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e129a3d-7c1a-400b-bf61-dfa92264c23d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:05.550534 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e129a3d-7c1a-400b-bf61-dfa92264c23d None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:05.572139 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 309/627] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Jun 18 04:59:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:05.572139 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e129a3d-7c1a-400b-bf61-dfa92264c23d None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:05.589632 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b35f2b3-6bd7-4bd8-864a-7cbc46fab190 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:05.591499 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b35f2b3-6bd7-4bd8-864a-7cbc46fab190 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:05.703703 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 319/628] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Jun 18 04:59:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 18 04:59:05.703901 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e129a3d-7c1a-400b-bf61-dfa92264c23d None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:05.704313 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b35f2b3-6bd7-4bd8-864a-7cbc46fab190 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:05.734570 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:05.745788 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc7b484-5c5f-4d39-adde-972b88d9f536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:05.756753 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc7b484-5c5f-4d39-adde-972b88d9f536 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:05.756753 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc7b484-5c5f-4d39-adde-972b88d9f536 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:05.756753 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc7b484-5c5f-4d39-adde-972b88d9f536 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:05.756753 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:05.760047 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc7b484-5c5f-4d39-adde-972b88d9f536 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 18 04:59:05.768251 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application [None req-fbc7b484-5c5f-4d39-adde-972b88d9f536 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 18 04:59:05.768251 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:05.768251 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:59:05.768251 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:59:05.768251 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:59:05.768251 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:59:05.768251 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:59:05.768251 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:59:05.768251 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:59:05.768251 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:59:05.768251 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:59:05.768251 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:59:05.768251 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 18 04:59:05.768251 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 18 04:59:05.768251 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 18 04:59:05.768251 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 18 04:59:05.770136 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 18 04:59:05.770136 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 18 04:59:05.770136 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 18 04:59:05.770136 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 18 04:59:05.770136 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:59:05.770136 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:59:05.770136 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:59:05.770136 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:59:05.770136 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:59:05.770136 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:59:05.770136 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:59:05.770136 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application with Lock( Jun 18 04:59:05.770136 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:59:05.770136 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:59:05.770136 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:59:05.770136 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:59:05.771122 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:59:05.771122 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:59:05.771122 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:59:05.771122 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:59:05.771122 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 18 04:59:05.771122 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 18 04:59:05.771122 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 18 04:59:05.771122 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 18 04:59:05.771122 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 18 04:59:05.771122 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 18 04:59:05.771122 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 18 04:59:05.771122 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 04:59:05.771122 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 310/629] 10.222.0.23 () {56 vars in 1102 bytes} [Sat Jun 18 04:59:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:59:05.771122 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8c3d3c-63d9-45ff-a8d3-7822c48cb526 None None] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:05.772013 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8c3d3c-63d9-45ff-a8d3-7822c48cb526 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:05.772013 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8c3d3c-63d9-45ff-a8d3-7822c48cb526 None None] PATH_INFO: `/` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:05.776759 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 311/630] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 04:59:05 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 04:59:05.777480 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8c3d3c-63d9-45ff-a8d3-7822c48cb526 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:05.824188 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b35f2b3-6bd7-4bd8-864a-7cbc46fab190 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:05.864533 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee87f5ac-018b-4e3e-892a-dd8242726150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:05.866142 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee87f5ac-018b-4e3e-892a-dd8242726150 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:05.866422 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee87f5ac-018b-4e3e-892a-dd8242726150 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:05.866693 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee87f5ac-018b-4e3e-892a-dd8242726150 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:05.873447 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee87f5ac-018b-4e3e-892a-dd8242726150 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:05.874905 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee87f5ac-018b-4e3e-892a-dd8242726150 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:05.886702 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8c3d3c-63d9-45ff-a8d3-7822c48cb526 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:05.888403 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-ee87f5ac-018b-4e3e-892a-dd8242726150 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 18 04:59:05.888403 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:05.888403 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:59:05.888403 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:59:05.888403 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:59:05.888403 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:59:05.888403 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:59:05.888403 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:59:05.888403 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:59:05.888403 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:59:05.888403 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:59:05.888403 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:59:05.888403 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 18 04:59:05.888403 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 18 04:59:05.888403 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 18 04:59:05.889309 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 18 04:59:05.889309 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:59:05.889309 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:59:05.889309 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:59:05.889309 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:59:05.889309 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:59:05.889309 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:59:05.889309 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:59:05.889309 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jun 18 04:59:05.889309 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:59:05.889309 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:59:05.889309 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:59:05.889309 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:59:05.889309 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:59:05.889309 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:59:05.889309 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:59:05.889309 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:59:05.890232 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 18 04:59:05.890232 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 18 04:59:05.890232 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 18 04:59:05.890232 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 18 04:59:05.890232 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 18 04:59:05.890232 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 18 04:59:05.890232 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 18 04:59:05.890232 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 04:59:05.893720 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 320/631] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Jun 18 04:59:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:59:05.894362 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee87f5ac-018b-4e3e-892a-dd8242726150 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:05.919559 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f59ba2-d1ef-4344-9622-8f3c66575003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:05.920432 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f59ba2-d1ef-4344-9622-8f3c66575003 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:05.920710 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f59ba2-d1ef-4344-9622-8f3c66575003 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:05.920978 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f59ba2-d1ef-4344-9622-8f3c66575003 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:05.931849 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f59ba2-d1ef-4344-9622-8f3c66575003 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:05.933268 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f59ba2-d1ef-4344-9622-8f3c66575003 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:05.950051 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application [None req-59f59ba2-d1ef-4344-9622-8f3c66575003 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 04:59:05.950051 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:05.950051 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:59:05.950051 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:59:05.950051 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:59:05.950051 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:59:05.950051 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:59:05.950051 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:59:05.950051 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:59:05.950051 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:59:05.950051 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:59:05.950051 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:59:05.950051 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 18 04:59:05.950051 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 18 04:59:05.950051 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 18 04:59:05.950992 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 18 04:59:05.950992 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:59:05.950992 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:59:05.950992 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:59:05.950992 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:59:05.950992 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:59:05.950992 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:59:05.950992 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:59:05.950992 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application with Lock( Jun 18 04:59:05.950992 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:59:05.950992 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:59:05.950992 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:59:05.950992 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:59:05.950992 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:59:05.950992 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:59:05.950992 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:59:05.950992 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:59:05.951908 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 18 04:59:05.951908 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 18 04:59:05.951908 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 04:59:05.951908 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 04:59:05.951908 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 312/632] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Jun 18 04:59:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:59:05.952459 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f3f8d5-7cdd-4a56-9701-b1c0f5a19ee6 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:05.952688 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f3f8d5-7cdd-4a56-9701-b1c0f5a19ee6 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:05.952958 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f3f8d5-7cdd-4a56-9701-b1c0f5a19ee6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:05.969253 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-53f3f8d5-7cdd-4a56-9701-b1c0f5a19ee6 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 04:59:05.970468 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-53f3f8d5-7cdd-4a56-9701-b1c0f5a19ee6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 04:59:05.997178 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee87f5ac-018b-4e3e-892a-dd8242726150 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:06.029754 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e5d5bb-4d29-4c00-b885-81ddf309bfdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:06.030413 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e5d5bb-4d29-4c00-b885-81ddf309bfdd None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:06.030700 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e5d5bb-4d29-4c00-b885-81ddf309bfdd None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:06.030979 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e5d5bb-4d29-4c00-b885-81ddf309bfdd None admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:06.031720 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e5d5bb-4d29-4c00-b885-81ddf309bfdd None admin] RBAC: Adding query filter params (name=service) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:06.032033 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e5d5bb-4d29-4c00-b885-81ddf309bfdd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:06.033440 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e5d5bb-4d29-4c00-b885-81ddf309bfdd None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:06.034475 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 313/633] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 04:59:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 18 04:59:06.035838 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a9708e-0fdc-43f2-a445-f9bd3fa4d379 None None] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:06.036125 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a9708e-0fdc-43f2-a445-f9bd3fa4d379 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:06.036396 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a9708e-0fdc-43f2-a445-f9bd3fa4d379 None None] PATH_INFO: `/` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:06.037698 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 314/634] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 04:59:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 04:59:06.041689 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a9708e-0fdc-43f2-a445-f9bd3fa4d379 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:06.053287 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 321/635] 10.222.0.23 () {56 vars in 1103 bytes} [Sat Jun 18 04:59:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:06.058542 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e5d5bb-4d29-4c00-b885-81ddf309bfdd None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:06.110618 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:06.137551 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:06.140688 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a9708e-0fdc-43f2-a445-f9bd3fa4d379 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:06.156736 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e5d5bb-4d29-4c00-b885-81ddf309bfdd None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:06.174928 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e000a095-a8a3-4d38-8ad4-7886cd20de2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:06.175570 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e000a095-a8a3-4d38-8ad4-7886cd20de2e None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:06.175850 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e000a095-a8a3-4d38-8ad4-7886cd20de2e None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:06.176123 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e000a095-a8a3-4d38-8ad4-7886cd20de2e None admin] PATH_INFO: `/v3/projects` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:06.176896 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e000a095-a8a3-4d38-8ad4-7886cd20de2e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:06.177212 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e000a095-a8a3-4d38-8ad4-7886cd20de2e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:06.179839 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e000a095-a8a3-4d38-8ad4-7886cd20de2e None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:06.191570 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 315/636] 10.222.0.23 () {56 vars in 1148 bytes} [Sat Jun 18 04:59:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:06.192740 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff54b63-f17e-40da-b996-acaddafbb442 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:06.193027 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff54b63-f17e-40da-b996-acaddafbb442 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:06.194397 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff54b63-f17e-40da-b996-acaddafbb442 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:06.197825 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b443523-0d22-4fc7-95cd-97c239e76b03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:06.198485 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b443523-0d22-4fc7-95cd-97c239e76b03 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:06.198767 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b443523-0d22-4fc7-95cd-97c239e76b03 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:06.199046 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b443523-0d22-4fc7-95cd-97c239e76b03 None admin] PATH_INFO: `/v3/domains` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:06.199765 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b443523-0d22-4fc7-95cd-97c239e76b03 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:06.200074 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b443523-0d22-4fc7-95cd-97c239e76b03 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:06.201472 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b443523-0d22-4fc7-95cd-97c239e76b03 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:06.210298 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-0ff54b63-f17e-40da-b996-acaddafbb442 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 04:59:06.211490 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-0ff54b63-f17e-40da-b996-acaddafbb442 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 04:59:06.213426 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 322/637] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Jun 18 04:59:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:06.214120 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b443523-0d22-4fc7-95cd-97c239e76b03 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:06.229617 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:06.256389 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:06.284271 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 316/638] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 04:59:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 18 04:59:06.288459 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff54b63-f17e-40da-b996-acaddafbb442 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:06.334295 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b443523-0d22-4fc7-95cd-97c239e76b03 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:06.368265 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3945ebf9-6dcb-4761-95af-33533fb6b3a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:06.369033 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3945ebf9-6dcb-4761-95af-33533fb6b3a5 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:06.369311 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3945ebf9-6dcb-4761-95af-33533fb6b3a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:06.369615 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3945ebf9-6dcb-4761-95af-33533fb6b3a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:06.381038 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3945ebf9-6dcb-4761-95af-33533fb6b3a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:06.383047 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3945ebf9-6dcb-4761-95af-33533fb6b3a5 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:06.385111 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff54b63-f17e-40da-b996-acaddafbb442 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:06.404259 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-3945ebf9-6dcb-4761-95af-33533fb6b3a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 04:59:06.404259 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:06.404259 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:59:06.404259 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:59:06.404259 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:59:06.404259 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:59:06.404259 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:59:06.404259 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:59:06.404259 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:59:06.404259 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:59:06.404259 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:59:06.404259 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:59:06.404259 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 18 04:59:06.404259 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 18 04:59:06.404259 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 18 04:59:06.405334 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 18 04:59:06.405334 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:59:06.405334 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:59:06.405334 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:59:06.405334 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:59:06.405334 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:59:06.405334 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:59:06.405334 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:59:06.405334 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jun 18 04:59:06.405334 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:59:06.405334 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:59:06.405334 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:59:06.405334 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:59:06.405334 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:59:06.405334 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:59:06.405334 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:59:06.405334 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:59:06.411022 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 18 04:59:06.411022 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 18 04:59:06.411022 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 04:59:06.411022 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 04:59:06.411022 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 323/639] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Jun 18 04:59:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:59:06.411022 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3945ebf9-6dcb-4761-95af-33533fb6b3a5 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:06.418914 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a788f02e-e4fb-4af0-bfa5-873127d4045b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:06.419705 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a788f02e-e4fb-4af0-bfa5-873127d4045b None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:06.419988 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a788f02e-e4fb-4af0-bfa5-873127d4045b None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:06.420263 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a788f02e-e4fb-4af0-bfa5-873127d4045b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:06.430290 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a788f02e-e4fb-4af0-bfa5-873127d4045b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 18 04:59:06.437896 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application [None req-a788f02e-e4fb-4af0-bfa5-873127d4045b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 18 04:59:06.437896 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:06.437896 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:59:06.437896 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:59:06.437896 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:59:06.437896 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:59:06.437896 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:59:06.437896 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:59:06.437896 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:59:06.437896 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:59:06.437896 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:59:06.437896 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:59:06.437896 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 18 04:59:06.437896 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 18 04:59:06.437896 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 18 04:59:06.437896 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 18 04:59:06.438872 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 18 04:59:06.438872 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 18 04:59:06.438872 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 18 04:59:06.438872 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 18 04:59:06.438872 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:59:06.438872 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:59:06.438872 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:59:06.438872 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:59:06.438872 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:59:06.438872 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:59:06.438872 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:59:06.438872 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application with Lock( Jun 18 04:59:06.438872 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:59:06.438872 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:59:06.438872 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:59:06.438872 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:59:06.439685 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:59:06.439685 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:59:06.439685 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:59:06.439685 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:59:06.439685 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 18 04:59:06.439685 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 18 04:59:06.439685 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 18 04:59:06.439685 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 18 04:59:06.439685 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 18 04:59:06.439685 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 18 04:59:06.439685 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 18 04:59:06.439685 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 04:59:06.439685 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 317/640] 10.222.0.23 () {56 vars in 1102 bytes} [Sat Jun 18 04:59:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:59:06.441283 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a788f02e-e4fb-4af0-bfa5-873127d4045b None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:06.471881 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:06.474152 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:06.509957 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3945ebf9-6dcb-4761-95af-33533fb6b3a5 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:06.543265 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a788f02e-e4fb-4af0-bfa5-873127d4045b None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:06.544648 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54295db1-ffbf-496d-9026-c3bde622e0e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:06.545578 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54295db1-ffbf-496d-9026-c3bde622e0e9 None admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:06.545863 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54295db1-ffbf-496d-9026-c3bde622e0e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:06.546136 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54295db1-ffbf-496d-9026-c3bde622e0e9 None admin] PATH_INFO: `/v3/projects/c95f8f3084c5472cb8364b7c57207394/users/16b49e866398400580ae48aa5a96fc04/roles/5ec7b68b793d4a54abd30460961346b9` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:06.560298 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54295db1-ffbf-496d-9026-c3bde622e0e9 None admin] RBAC: Authorizing `identity:create_grant(project_id=c95f8f3084c5472cb8364b7c57207394, user_id=16b49e866398400580ae48aa5a96fc04, role_id=5ec7b68b793d4a54abd30460961346b9)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:06.561776 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54295db1-ffbf-496d-9026-c3bde622e0e9 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:06.576552 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534c3a4a-646d-4de8-a65b-398f87d69d77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:06.577333 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534c3a4a-646d-4de8-a65b-398f87d69d77 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:06.577646 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534c3a4a-646d-4de8-a65b-398f87d69d77 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:06.577971 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534c3a4a-646d-4de8-a65b-398f87d69d77 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:06.585204 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534c3a4a-646d-4de8-a65b-398f87d69d77 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:06.586659 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534c3a4a-646d-4de8-a65b-398f87d69d77 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:06.600555 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application [None req-534c3a4a-646d-4de8-a65b-398f87d69d77 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 18 04:59:06.600555 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:06.600555 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:59:06.600555 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:59:06.600555 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:59:06.600555 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:59:06.600555 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:59:06.600555 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:59:06.600555 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:59:06.600555 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:59:06.600555 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:59:06.600555 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:59:06.600555 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 18 04:59:06.600555 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 18 04:59:06.600555 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 18 04:59:06.601708 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 18 04:59:06.601708 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:59:06.601708 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:59:06.601708 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:59:06.601708 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:59:06.601708 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:59:06.601708 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:59:06.601708 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:59:06.601708 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application with Lock( Jun 18 04:59:06.601708 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:59:06.601708 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:59:06.601708 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:59:06.601708 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:59:06.601708 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:59:06.601708 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:59:06.601708 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:59:06.601708 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:59:06.602637 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 18 04:59:06.602637 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 18 04:59:06.602637 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 18 04:59:06.602637 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 18 04:59:06.602637 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 18 04:59:06.602637 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 18 04:59:06.602637 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 18 04:59:06.602637 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 04:59:06.602637 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 318/641] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Jun 18 04:59:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:59:06.602637 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534c3a4a-646d-4de8-a65b-398f87d69d77 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:06.630450 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 324/642] 10.222.0.23 () {58 vars in 1439 bytes} [Sat Jun 18 04:59:06 2022] PUT /identity/v3/projects/c95f8f3084c5472cb8364b7c57207394/users/16b49e866398400580ae48aa5a96fc04/roles/5ec7b68b793d4a54abd30460961346b9 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 04:59:06.632432 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54295db1-ffbf-496d-9026-c3bde622e0e9 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:06.710508 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:06.721002 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534c3a4a-646d-4de8-a65b-398f87d69d77 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:06.762212 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54295db1-ffbf-496d-9026-c3bde622e0e9 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:06.763129 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81148d76-34e3-456f-bed7-55cc101e43c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:06.764034 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81148d76-34e3-456f-bed7-55cc101e43c6 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:06.764409 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81148d76-34e3-456f-bed7-55cc101e43c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:06.764809 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81148d76-34e3-456f-bed7-55cc101e43c6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:06.779463 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:06.781570 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81148d76-34e3-456f-bed7-55cc101e43c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 18 04:59:06.790452 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application [None req-81148d76-34e3-456f-bed7-55cc101e43c6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 18 04:59:06.790452 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:06.790452 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:59:06.790452 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:59:06.790452 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:59:06.790452 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:59:06.790452 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:59:06.790452 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:59:06.790452 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:59:06.790452 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:59:06.790452 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:59:06.790452 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:59:06.790452 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 18 04:59:06.790452 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 18 04:59:06.790452 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 18 04:59:06.790452 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 18 04:59:06.794573 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 18 04:59:06.794573 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 18 04:59:06.794573 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 18 04:59:06.794573 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 18 04:59:06.794573 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:59:06.794573 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:59:06.794573 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:59:06.794573 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:59:06.794573 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:59:06.794573 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:59:06.794573 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:59:06.794573 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application with Lock( Jun 18 04:59:06.794573 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:59:06.794573 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:59:06.794573 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:59:06.794573 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:59:06.795659 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:59:06.795659 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:59:06.795659 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:59:06.795659 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:59:06.795659 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 18 04:59:06.795659 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 18 04:59:06.795659 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 18 04:59:06.795659 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 18 04:59:06.795659 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 18 04:59:06.795659 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 18 04:59:06.795659 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 18 04:59:06.795659 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 04:59:06.795659 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 319/643] 10.222.0.23 () {56 vars in 1102 bytes} [Sat Jun 18 04:59:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:59:06.795659 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81148d76-34e3-456f-bed7-55cc101e43c6 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:06.816014 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f414489c-97a4-4b67-9ad4-a2812449c2cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:06.816749 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f414489c-97a4-4b67-9ad4-a2812449c2cd None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:06.817030 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f414489c-97a4-4b67-9ad4-a2812449c2cd None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:06.818643 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f414489c-97a4-4b67-9ad4-a2812449c2cd None admin] PATH_INFO: `/v3/domains` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:06.818643 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f414489c-97a4-4b67-9ad4-a2812449c2cd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:06.818960 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f414489c-97a4-4b67-9ad4-a2812449c2cd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:06.820342 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f414489c-97a4-4b67-9ad4-a2812449c2cd None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:06.832013 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 325/644] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Jun 18 04:59:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:06.832613 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f414489c-97a4-4b67-9ad4-a2812449c2cd None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:06.912393 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81148d76-34e3-456f-bed7-55cc101e43c6 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:06.946081 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f414489c-97a4-4b67-9ad4-a2812449c2cd None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:06.952007 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836c62d4-14c9-4dc3-88ec-96126b4e0506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:06.952674 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836c62d4-14c9-4dc3-88ec-96126b4e0506 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:06.952960 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836c62d4-14c9-4dc3-88ec-96126b4e0506 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:06.953242 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836c62d4-14c9-4dc3-88ec-96126b4e0506 None admin] PATH_INFO: `/v3/roles` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:06.954016 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836c62d4-14c9-4dc3-88ec-96126b4e0506 None admin] RBAC: Adding query filter params (name=service) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:06.954332 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836c62d4-14c9-4dc3-88ec-96126b4e0506 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:06.955693 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836c62d4-14c9-4dc3-88ec-96126b4e0506 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:06.968819 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 320/645] 10.222.0.23 () {56 vars in 1103 bytes} [Sat Jun 18 04:59:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:06.969725 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836c62d4-14c9-4dc3-88ec-96126b4e0506 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:06.985722 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:06.989026 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d074796-d8c8-4b10-a425-23f62d3f991e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:06.989724 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d074796-d8c8-4b10-a425-23f62d3f991e None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:06.990799 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d074796-d8c8-4b10-a425-23f62d3f991e None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:06.991260 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d074796-d8c8-4b10-a425-23f62d3f991e None admin] PATH_INFO: `/v3/projects` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:06.991973 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d074796-d8c8-4b10-a425-23f62d3f991e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:06.992287 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d074796-d8c8-4b10-a425-23f62d3f991e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:06.993782 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d074796-d8c8-4b10-a425-23f62d3f991e None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:07.011108 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 326/646] 10.222.0.23 () {56 vars in 1148 bytes} [Sat Jun 18 04:59:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:07.011108 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d074796-d8c8-4b10-a425-23f62d3f991e None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:07.044306 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:07.083479 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836c62d4-14c9-4dc3-88ec-96126b4e0506 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:07.118552 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d074796-d8c8-4b10-a425-23f62d3f991e None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:07.123733 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355acd8e-7a30-4c10-a0b5-c749fea32988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:07.124393 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355acd8e-7a30-4c10-a0b5-c749fea32988 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:07.124674 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355acd8e-7a30-4c10-a0b5-c749fea32988 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:07.124944 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355acd8e-7a30-4c10-a0b5-c749fea32988 None admin] PATH_INFO: `/v3/roles` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:07.125709 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355acd8e-7a30-4c10-a0b5-c749fea32988 None admin] RBAC: Adding query filter params (name=service) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:07.126016 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355acd8e-7a30-4c10-a0b5-c749fea32988 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:07.127367 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355acd8e-7a30-4c10-a0b5-c749fea32988 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:07.137599 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 321/647] 10.222.0.23 () {56 vars in 1103 bytes} [Sat Jun 18 04:59:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:07.137790 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355acd8e-7a30-4c10-a0b5-c749fea32988 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:07.163947 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13fb3a9-da6b-4124-be8a-d00ecce20171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:07.165029 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13fb3a9-da6b-4124-be8a-d00ecce20171 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:07.165525 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13fb3a9-da6b-4124-be8a-d00ecce20171 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:07.165964 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13fb3a9-da6b-4124-be8a-d00ecce20171 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:07.173973 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13fb3a9-da6b-4124-be8a-d00ecce20171 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:07.175595 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13fb3a9-da6b-4124-be8a-d00ecce20171 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:07.191427 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-a13fb3a9-da6b-4124-be8a-d00ecce20171 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 18 04:59:07.191427 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:07.191427 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:59:07.191427 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:59:07.191427 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:59:07.191427 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:59:07.191427 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:59:07.191427 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:59:07.191427 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:59:07.191427 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:59:07.191427 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:59:07.191427 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:59:07.191427 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 18 04:59:07.191427 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 18 04:59:07.191427 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 18 04:59:07.191427 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 18 04:59:07.192628 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:59:07.192628 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:59:07.192628 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 18 04:59:07.192628 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 18 04:59:07.192628 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 18 04:59:07.192628 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 18 04:59:07.192628 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:59:07.192628 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:59:07.192628 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:59:07.192628 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:59:07.192628 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:59:07.192628 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jun 18 04:59:07.192628 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:59:07.192628 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:59:07.192628 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:59:07.192628 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:59:07.192628 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:59:07.192628 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:59:07.194341 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:59:07.194341 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:59:07.194341 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 18 04:59:07.194341 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 18 04:59:07.194341 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 18 04:59:07.194341 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 18 04:59:07.194341 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 18 04:59:07.194341 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 18 04:59:07.194341 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 18 04:59:07.194341 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 04:59:07.196632 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 327/648] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Jun 18 04:59:07 2022] GET /identity/v3/users/glance => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:59:07.197671 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13fb3a9-da6b-4124-be8a-d00ecce20171 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:07.227956 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:07.253535 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355acd8e-7a30-4c10-a0b5-c749fea32988 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:07.292628 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93aff0c6-3584-48b4-9d63-9dfa4c1e17f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:07.293758 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aff0c6-3584-48b4-9d63-9dfa4c1e17f3 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:07.294050 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aff0c6-3584-48b4-9d63-9dfa4c1e17f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:07.294380 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aff0c6-3584-48b4-9d63-9dfa4c1e17f3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:07.301833 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:07.309689 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93aff0c6-3584-48b4-9d63-9dfa4c1e17f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:07.330401 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13fb3a9-da6b-4124-be8a-d00ecce20171 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:07.330562 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93aff0c6-3584-48b4-9d63-9dfa4c1e17f3 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:07.334261 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application [None req-93aff0c6-3584-48b4-9d63-9dfa4c1e17f3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 04:59:07.334261 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:07.334261 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:59:07.334261 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:59:07.334261 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:59:07.334261 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:59:07.334261 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:59:07.334261 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:59:07.334261 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:59:07.334261 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:59:07.334261 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:59:07.334261 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:59:07.334261 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 18 04:59:07.334261 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 18 04:59:07.334261 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 18 04:59:07.335392 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 18 04:59:07.335392 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:59:07.335392 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:59:07.335392 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:59:07.335392 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:59:07.335392 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:59:07.335392 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:59:07.335392 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:59:07.335392 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application with Lock( Jun 18 04:59:07.335392 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:59:07.335392 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:59:07.335392 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:59:07.335392 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:59:07.335392 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:59:07.335392 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:59:07.335392 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:59:07.335392 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:59:07.336461 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 18 04:59:07.336461 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 18 04:59:07.336461 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 04:59:07.336461 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 04:59:07.339916 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 322/649] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Jun 18 04:59:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:59:07.340798 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93aff0c6-3584-48b4-9d63-9dfa4c1e17f3 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:07.348813 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268f06dc-3a1d-4a15-86ff-bc350df69562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:07.350347 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268f06dc-3a1d-4a15-86ff-bc350df69562 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:07.350770 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268f06dc-3a1d-4a15-86ff-bc350df69562 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:07.351177 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268f06dc-3a1d-4a15-86ff-bc350df69562 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:07.352393 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268f06dc-3a1d-4a15-86ff-bc350df69562 None admin] RBAC: Adding query filter params (role.id=5ec7b68b793d4a54abd30460961346b9, scope.project.id=c95f8f3084c5472cb8364b7c57207394, user.id=15e7f45ee0744c1d9602187eac8e8b80) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:07.352802 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268f06dc-3a1d-4a15-86ff-bc350df69562 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:07.355016 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268f06dc-3a1d-4a15-86ff-bc350df69562 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:07.371282 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 328/650] 10.222.0.23 () {56 vars in 1374 bytes} [Sat Jun 18 04:59:07 2022] GET /identity/v3/role_assignments?user.id=15e7f45ee0744c1d9602187eac8e8b80&scope.project.id=c95f8f3084c5472cb8364b7c57207394&role.id=5ec7b68b793d4a54abd30460961346b9 => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:07.372664 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268f06dc-3a1d-4a15-86ff-bc350df69562 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:07.463027 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93aff0c6-3584-48b4-9d63-9dfa4c1e17f3 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:07.497838 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:07.502327 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0939037f-15b1-4ff4-8ce9-66cf181c8ee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:07.503141 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0939037f-15b1-4ff4-8ce9-66cf181c8ee5 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:07.503429 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0939037f-15b1-4ff4-8ce9-66cf181c8ee5 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:07.503709 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0939037f-15b1-4ff4-8ce9-66cf181c8ee5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:07.519812 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0939037f-15b1-4ff4-8ce9-66cf181c8ee5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:07.521873 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0939037f-15b1-4ff4-8ce9-66cf181c8ee5 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:07.561124 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application [None req-0939037f-15b1-4ff4-8ce9-66cf181c8ee5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 04:59:07.561124 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:07.561124 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:59:07.561124 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:59:07.561124 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:59:07.561124 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:59:07.561124 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:59:07.561124 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:59:07.561124 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:59:07.561124 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:59:07.561124 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:59:07.561124 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:59:07.561124 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 18 04:59:07.561124 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 18 04:59:07.561124 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 18 04:59:07.562076 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 18 04:59:07.562076 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:59:07.562076 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:59:07.562076 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:59:07.562076 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:59:07.562076 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:59:07.562076 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:59:07.562076 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:59:07.562076 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application with Lock( Jun 18 04:59:07.562076 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:59:07.562076 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:59:07.562076 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:59:07.562076 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:59:07.562076 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:59:07.562076 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:59:07.562076 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:59:07.562076 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:59:07.562966 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 18 04:59:07.562966 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 18 04:59:07.562966 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 04:59:07.562966 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 04:59:07.562966 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 323/651] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Jun 18 04:59:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:59:07.563846 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0939037f-15b1-4ff4-8ce9-66cf181c8ee5 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:07.600018 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268f06dc-3a1d-4a15-86ff-bc350df69562 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:07.655806 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6e7811-a7aa-41e2-af13-dda31523cc51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:07.656496 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6e7811-a7aa-41e2-af13-dda31523cc51 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:07.656777 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6e7811-a7aa-41e2-af13-dda31523cc51 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:07.657211 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6e7811-a7aa-41e2-af13-dda31523cc51 None admin] PATH_INFO: `/v3/users` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:07.657951 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6e7811-a7aa-41e2-af13-dda31523cc51 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:07.658757 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6e7811-a7aa-41e2-af13-dda31523cc51 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:07.660299 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6e7811-a7aa-41e2-af13-dda31523cc51 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:07.690542 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 329/652] 10.222.0.23 () {56 vars in 1137 bytes} [Sat Jun 18 04:59:07 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:07.691689 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6e7811-a7aa-41e2-af13-dda31523cc51 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:07.718605 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0939037f-15b1-4ff4-8ce9-66cf181c8ee5 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:07.721074 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:07.764083 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981e621c-7386-4d2f-a43d-37c2fc063624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:07.765019 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981e621c-7386-4d2f-a43d-37c2fc063624 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:07.765304 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981e621c-7386-4d2f-a43d-37c2fc063624 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:07.765609 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981e621c-7386-4d2f-a43d-37c2fc063624 None admin] PATH_INFO: `/v3/domains` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:07.767016 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981e621c-7386-4d2f-a43d-37c2fc063624 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:07.767349 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981e621c-7386-4d2f-a43d-37c2fc063624 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:07.769354 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981e621c-7386-4d2f-a43d-37c2fc063624 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:07.787794 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 324/653] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Jun 18 04:59:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:07.789231 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981e621c-7386-4d2f-a43d-37c2fc063624 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:07.813901 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6e7811-a7aa-41e2-af13-dda31523cc51 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:07.835814 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:07.859793 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f706f28b-b132-482a-8f0f-af608fc5c0d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:07.860974 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f706f28b-b132-482a-8f0f-af608fc5c0d8 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:07.861467 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f706f28b-b132-482a-8f0f-af608fc5c0d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:07.861990 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f706f28b-b132-482a-8f0f-af608fc5c0d8 None admin] PATH_INFO: `/v3/domains` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:07.863057 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f706f28b-b132-482a-8f0f-af608fc5c0d8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:07.864009 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f706f28b-b132-482a-8f0f-af608fc5c0d8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:07.866374 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f706f28b-b132-482a-8f0f-af608fc5c0d8 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:07.884352 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 330/654] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Jun 18 04:59:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:07.885945 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f706f28b-b132-482a-8f0f-af608fc5c0d8 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:07.901192 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981e621c-7386-4d2f-a43d-37c2fc063624 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:07.939158 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276786cc-a540-4670-a81f-6d7524df4919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:07.939800 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276786cc-a540-4670-a81f-6d7524df4919 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:07.940083 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276786cc-a540-4670-a81f-6d7524df4919 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:07.940350 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276786cc-a540-4670-a81f-6d7524df4919 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:07.984312 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:08.010174 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f706f28b-b132-482a-8f0f-af608fc5c0d8 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:08.020487 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276786cc-a540-4670-a81f-6d7524df4919 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:08.022254 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276786cc-a540-4670-a81f-6d7524df4919 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:08.049183 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcd7030-c3f2-461e-97bf-37ec81078b43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:08.050206 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcd7030-c3f2-461e-97bf-37ec81078b43 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:08.050593 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcd7030-c3f2-461e-97bf-37ec81078b43 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:08.051128 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcd7030-c3f2-461e-97bf-37ec81078b43 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:08.059798 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcd7030-c3f2-461e-97bf-37ec81078b43 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:08.061597 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcd7030-c3f2-461e-97bf-37ec81078b43 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:08.067631 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:08.078392 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-4bcd7030-c3f2-461e-97bf-37ec81078b43 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 18 04:59:08.078392 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:08.078392 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:59:08.078392 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:59:08.078392 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:59:08.078392 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:59:08.078392 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:59:08.078392 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:59:08.078392 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:59:08.078392 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:59:08.078392 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:59:08.078392 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:59:08.078392 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 18 04:59:08.078392 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 18 04:59:08.078392 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 18 04:59:08.078392 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 18 04:59:08.085645 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:59:08.085645 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:59:08.085645 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 18 04:59:08.085645 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 18 04:59:08.085645 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 18 04:59:08.085645 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 18 04:59:08.085645 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:59:08.085645 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:59:08.085645 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:59:08.085645 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:59:08.085645 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:59:08.085645 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jun 18 04:59:08.085645 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:59:08.085645 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:59:08.085645 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:59:08.085645 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:59:08.085645 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:59:08.085645 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:59:08.086801 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:59:08.086801 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:59:08.086801 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 18 04:59:08.086801 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 18 04:59:08.086801 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 18 04:59:08.086801 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 18 04:59:08.086801 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 18 04:59:08.086801 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 18 04:59:08.086801 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 18 04:59:08.086801 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 04:59:08.086801 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 331/655] 10.222.0.23 () {56 vars in 1102 bytes} [Sat Jun 18 04:59:07 2022] GET /identity/v3/users/neutron => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:59:08.086801 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcd7030-c3f2-461e-97bf-37ec81078b43 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:08.135708 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 325/656] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Jun 18 04:59:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 18 04:59:08.136840 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276786cc-a540-4670-a81f-6d7524df4919 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:08.196624 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcd7030-c3f2-461e-97bf-37ec81078b43 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:08.234428 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531a5499-da33-44ac-b5d1-24f1fa3da84a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:08.235644 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531a5499-da33-44ac-b5d1-24f1fa3da84a None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:08.235782 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531a5499-da33-44ac-b5d1-24f1fa3da84a None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:08.236172 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531a5499-da33-44ac-b5d1-24f1fa3da84a None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:08.238940 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276786cc-a540-4670-a81f-6d7524df4919 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:08.239432 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:08.247135 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531a5499-da33-44ac-b5d1-24f1fa3da84a None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:08.248945 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531a5499-da33-44ac-b5d1-24f1fa3da84a None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:08.265172 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-531a5499-da33-44ac-b5d1-24f1fa3da84a None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 18 04:59:08.265172 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:08.265172 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:59:08.265172 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:59:08.265172 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:59:08.265172 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:59:08.265172 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:59:08.265172 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:59:08.265172 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:59:08.265172 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:59:08.265172 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:59:08.265172 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:59:08.265172 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 18 04:59:08.265172 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 18 04:59:08.265172 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 18 04:59:08.265172 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 18 04:59:08.266263 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:59:08.266263 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:59:08.266263 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 18 04:59:08.266263 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 18 04:59:08.266263 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 18 04:59:08.266263 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 18 04:59:08.266263 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:59:08.266263 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:59:08.266263 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:59:08.266263 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:59:08.266263 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:59:08.266263 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jun 18 04:59:08.266263 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:59:08.266263 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:59:08.266263 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:59:08.266263 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:59:08.266263 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:59:08.266263 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:59:08.267365 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:59:08.267365 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:59:08.267365 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 18 04:59:08.267365 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 18 04:59:08.267365 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 18 04:59:08.267365 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 18 04:59:08.267365 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 18 04:59:08.267365 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 18 04:59:08.267365 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 18 04:59:08.267365 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 04:59:08.267365 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 332/657] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Jun 18 04:59:08 2022] GET /identity/v3/users/cinder => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:59:08.268739 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531a5499-da33-44ac-b5d1-24f1fa3da84a None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:08.281209 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816a5722-a7dc-4e55-8bf9-01852c99bbf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:08.282136 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816a5722-a7dc-4e55-8bf9-01852c99bbf9 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:08.282593 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816a5722-a7dc-4e55-8bf9-01852c99bbf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:08.283012 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816a5722-a7dc-4e55-8bf9-01852c99bbf9 None admin] PATH_INFO: `/v3/users` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:08.283987 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816a5722-a7dc-4e55-8bf9-01852c99bbf9 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:08.284447 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816a5722-a7dc-4e55-8bf9-01852c99bbf9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:08.286076 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816a5722-a7dc-4e55-8bf9-01852c99bbf9 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:08.295093 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:08.307497 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 326/658] 10.222.0.23 () {56 vars in 1139 bytes} [Sat Jun 18 04:59:08 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:08.308853 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816a5722-a7dc-4e55-8bf9-01852c99bbf9 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:08.387132 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531a5499-da33-44ac-b5d1-24f1fa3da84a None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:08.413698 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816a5722-a7dc-4e55-8bf9-01852c99bbf9 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:08.426434 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441100bc-987f-4637-b7dc-24743fd9f0d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:08.427235 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441100bc-987f-4637-b7dc-24743fd9f0d1 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:08.428114 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441100bc-987f-4637-b7dc-24743fd9f0d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:08.428395 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441100bc-987f-4637-b7dc-24743fd9f0d1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:08.440352 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441100bc-987f-4637-b7dc-24743fd9f0d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:08.441951 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441100bc-987f-4637-b7dc-24743fd9f0d1 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:08.451313 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d5197e-4b7e-46fc-8d3a-43086ceb732d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:08.452183 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d5197e-4b7e-46fc-8d3a-43086ceb732d None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:08.452631 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d5197e-4b7e-46fc-8d3a-43086ceb732d None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:08.453067 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d5197e-4b7e-46fc-8d3a-43086ceb732d None admin] PATH_INFO: `/v3/users` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:08.454086 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d5197e-4b7e-46fc-8d3a-43086ceb732d None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:08.454544 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d5197e-4b7e-46fc-8d3a-43086ceb732d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:08.456098 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d5197e-4b7e-46fc-8d3a-43086ceb732d None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:08.461432 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-441100bc-987f-4637-b7dc-24743fd9f0d1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 04:59:08.461432 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:08.461432 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:59:08.461432 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:59:08.461432 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:59:08.461432 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:59:08.461432 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:59:08.461432 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:59:08.461432 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:59:08.461432 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:59:08.461432 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:59:08.461432 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:59:08.461432 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 18 04:59:08.461432 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 18 04:59:08.461432 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 18 04:59:08.462486 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 18 04:59:08.462486 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:59:08.462486 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:59:08.462486 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:59:08.462486 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:59:08.462486 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:59:08.462486 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:59:08.462486 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:59:08.462486 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jun 18 04:59:08.462486 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:59:08.462486 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:59:08.462486 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:59:08.462486 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:59:08.462486 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:59:08.462486 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:59:08.462486 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:59:08.462486 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:59:08.463735 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 18 04:59:08.463735 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 18 04:59:08.463735 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 04:59:08.463735 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 04:59:08.463735 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 333/659] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Jun 18 04:59:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:59:08.463735 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441100bc-987f-4637-b7dc-24743fd9f0d1 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:08.477301 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 327/660] 10.222.0.23 () {56 vars in 1137 bytes} [Sat Jun 18 04:59:08 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:08.478445 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d5197e-4b7e-46fc-8d3a-43086ceb732d None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:08.518648 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:08.535520 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:08.576750 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441100bc-987f-4637-b7dc-24743fd9f0d1 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:08.583240 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d5197e-4b7e-46fc-8d3a-43086ceb732d None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:08.615759 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6df505e-262f-445a-ae40-dbf696d1ea20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:08.617275 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6df505e-262f-445a-ae40-dbf696d1ea20 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:08.617736 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6df505e-262f-445a-ae40-dbf696d1ea20 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:08.618181 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6df505e-262f-445a-ae40-dbf696d1ea20 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:08.622648 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810c8e5b-5db7-4660-be6a-ec67727d7766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:08.623010 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810c8e5b-5db7-4660-be6a-ec67727d7766 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:08.623301 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810c8e5b-5db7-4660-be6a-ec67727d7766 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:08.623576 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810c8e5b-5db7-4660-be6a-ec67727d7766 None admin] PATH_INFO: `/v3/domains` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:08.624301 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810c8e5b-5db7-4660-be6a-ec67727d7766 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:08.624613 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810c8e5b-5db7-4660-be6a-ec67727d7766 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:08.626037 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810c8e5b-5db7-4660-be6a-ec67727d7766 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:08.639376 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 328/661] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Jun 18 04:59:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:08.640316 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810c8e5b-5db7-4660-be6a-ec67727d7766 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:08.698497 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6df505e-262f-445a-ae40-dbf696d1ea20 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:08.700226 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6df505e-262f-445a-ae40-dbf696d1ea20 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:08.753417 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:08.754965 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810c8e5b-5db7-4660-be6a-ec67727d7766 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:08.769921 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:08.792114 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0c3f50-cce4-43f1-a7aa-d044583ee809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:08.792762 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0c3f50-cce4-43f1-a7aa-d044583ee809 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:08.793048 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0c3f50-cce4-43f1-a7aa-d044583ee809 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:08.793325 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0c3f50-cce4-43f1-a7aa-d044583ee809 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:08.816921 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 334/662] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Jun 18 04:59:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 18 04:59:08.818701 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6df505e-262f-445a-ae40-dbf696d1ea20 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:08.868343 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0c3f50-cce4-43f1-a7aa-d044583ee809 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:08.869844 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0c3f50-cce4-43f1-a7aa-d044583ee809 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:08.924089 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6df505e-262f-445a-ae40-dbf696d1ea20 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:08.960842 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60811765-6436-45b1-97a0-1bbfb1e92a70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:08.961666 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60811765-6436-45b1-97a0-1bbfb1e92a70 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:08.961948 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60811765-6436-45b1-97a0-1bbfb1e92a70 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:08.962223 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60811765-6436-45b1-97a0-1bbfb1e92a70 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:08.969023 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60811765-6436-45b1-97a0-1bbfb1e92a70 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:08.970494 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60811765-6436-45b1-97a0-1bbfb1e92a70 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:08.978847 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 329/663] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Jun 18 04:59:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 18 04:59:08.978847 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943d0d21-6c50-4eb3-9823-941a26193d12 None None] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:08.978847 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943d0d21-6c50-4eb3-9823-941a26193d12 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:08.978847 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943d0d21-6c50-4eb3-9823-941a26193d12 None None] PATH_INFO: `/` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:08.978847 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 330/664] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 04:59:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 04:59:08.978847 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943d0d21-6c50-4eb3-9823-941a26193d12 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:08.991026 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-60811765-6436-45b1-97a0-1bbfb1e92a70 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 18 04:59:08.991026 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:08.991026 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:59:08.991026 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:59:08.991026 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:59:08.991026 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:59:08.991026 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:59:08.991026 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:59:08.991026 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:59:08.991026 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:59:08.991026 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:59:08.991026 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:59:08.991026 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 18 04:59:08.991026 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 18 04:59:08.991026 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 18 04:59:08.991958 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 18 04:59:08.991958 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:59:08.991958 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:59:08.991958 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:59:08.991958 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:59:08.991958 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:59:08.991958 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:59:08.991958 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:59:08.991958 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jun 18 04:59:08.991958 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:59:08.991958 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:59:08.991958 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:59:08.991958 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:59:08.991958 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:59:08.991958 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:59:08.991958 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:59:08.991958 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:59:08.992870 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 18 04:59:08.992870 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 18 04:59:08.992870 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 18 04:59:08.992870 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 18 04:59:08.992870 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 18 04:59:08.992870 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 18 04:59:08.992870 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 18 04:59:08.992870 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 04:59:08.992870 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 335/665] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Jun 18 04:59:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:59:08.993356 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60811765-6436-45b1-97a0-1bbfb1e92a70 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:09.000517 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:09.005621 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:09.099087 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943d0d21-6c50-4eb3-9823-941a26193d12 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:09.103100 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60811765-6436-45b1-97a0-1bbfb1e92a70 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:09.136387 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e0a1fb-80aa-48f4-9f72-9ad6b98c7b1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:09.137265 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e0a1fb-80aa-48f4-9f72-9ad6b98c7b1b None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:09.137583 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e0a1fb-80aa-48f4-9f72-9ad6b98c7b1b None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:09.137865 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e0a1fb-80aa-48f4-9f72-9ad6b98c7b1b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:09.138696 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7514bfaf-9c34-44c3-b824-7657cd053b66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:09.139460 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7514bfaf-9c34-44c3-b824-7657cd053b66 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:09.139740 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7514bfaf-9c34-44c3-b824-7657cd053b66 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:09.140028 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7514bfaf-9c34-44c3-b824-7657cd053b66 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:09.149864 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e0a1fb-80aa-48f4-9f72-9ad6b98c7b1b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:09.151312 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e0a1fb-80aa-48f4-9f72-9ad6b98c7b1b None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:09.152492 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7514bfaf-9c34-44c3-b824-7657cd053b66 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:09.153996 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7514bfaf-9c34-44c3-b824-7657cd053b66 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:09.171298 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-7514bfaf-9c34-44c3-b824-7657cd053b66 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 04:59:09.171298 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:09.171298 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:59:09.171298 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:59:09.171298 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:59:09.171298 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:59:09.171298 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:59:09.171298 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:59:09.171298 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:59:09.171298 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:59:09.171298 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:59:09.171298 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:59:09.171298 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 18 04:59:09.171298 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 18 04:59:09.171298 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 18 04:59:09.172256 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 18 04:59:09.172256 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:59:09.172256 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:59:09.172256 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:59:09.172256 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:59:09.172256 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:59:09.172256 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:59:09.172256 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:59:09.172256 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jun 18 04:59:09.172256 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:59:09.172256 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:59:09.172256 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:59:09.172256 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:59:09.172256 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:59:09.172256 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:59:09.172256 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:59:09.172256 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:59:09.173149 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 18 04:59:09.173149 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 18 04:59:09.173149 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 04:59:09.173149 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 04:59:09.173370 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application [None req-41e0a1fb-80aa-48f4-9f72-9ad6b98c7b1b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 04:59:09.173370 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:09.173370 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:59:09.173370 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:59:09.173370 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:59:09.173370 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:59:09.173370 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:59:09.173370 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:59:09.173370 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:59:09.173370 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:59:09.173370 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:59:09.173370 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:59:09.173370 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 18 04:59:09.173370 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 18 04:59:09.173370 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 18 04:59:09.174245 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 18 04:59:09.174245 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:59:09.174245 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:59:09.174245 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:59:09.174245 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:59:09.174245 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:59:09.174245 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:59:09.174245 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:59:09.174245 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application with Lock( Jun 18 04:59:09.174245 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:59:09.174245 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:59:09.174245 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:59:09.174245 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:59:09.174245 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:59:09.174245 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:59:09.174245 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:59:09.174245 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:59:09.175145 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 18 04:59:09.175145 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 18 04:59:09.175145 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 04:59:09.175145 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 04:59:09.175371 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 336/666] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Jun 18 04:59:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:59:09.175371 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e854108-7fef-4cdd-8e91-1da64c6b182e None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:09.175371 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e854108-7fef-4cdd-8e91-1da64c6b182e None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:09.175371 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e854108-7fef-4cdd-8e91-1da64c6b182e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:09.177294 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 331/667] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Jun 18 04:59:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:59:09.178783 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e0a1fb-80aa-48f4-9f72-9ad6b98c7b1b None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:09.197272 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-2e854108-7fef-4cdd-8e91-1da64c6b182e None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 04:59:09.199202 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-2e854108-7fef-4cdd-8e91-1da64c6b182e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 04:59:09.249892 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:09.264680 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:09.277445 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 337/668] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 04:59:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 18 04:59:09.277445 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e854108-7fef-4cdd-8e91-1da64c6b182e None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:09.295843 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e0a1fb-80aa-48f4-9f72-9ad6b98c7b1b None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:09.330947 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85852e33-70e2-45d8-bba9-477ceb9800b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:09.331599 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85852e33-70e2-45d8-bba9-477ceb9800b5 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:09.331892 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85852e33-70e2-45d8-bba9-477ceb9800b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:09.332168 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85852e33-70e2-45d8-bba9-477ceb9800b5 None admin] PATH_INFO: `/v3/projects` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:09.332945 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85852e33-70e2-45d8-bba9-477ceb9800b5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:09.333258 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85852e33-70e2-45d8-bba9-477ceb9800b5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:09.335236 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85852e33-70e2-45d8-bba9-477ceb9800b5 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:09.346786 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 332/669] 10.222.0.23 () {56 vars in 1148 bytes} [Sat Jun 18 04:59:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:09.347559 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85852e33-70e2-45d8-bba9-477ceb9800b5 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:09.400857 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e854108-7fef-4cdd-8e91-1da64c6b182e None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:09.437918 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6264544-f8b8-406e-aa1e-e39c8d5e17e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:09.438577 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6264544-f8b8-406e-aa1e-e39c8d5e17e2 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:09.438861 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6264544-f8b8-406e-aa1e-e39c8d5e17e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:09.439143 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6264544-f8b8-406e-aa1e-e39c8d5e17e2 None admin] PATH_INFO: `/v3/domains` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:09.439986 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6264544-f8b8-406e-aa1e-e39c8d5e17e2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:09.440416 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6264544-f8b8-406e-aa1e-e39c8d5e17e2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:09.441823 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6264544-f8b8-406e-aa1e-e39c8d5e17e2 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:09.454621 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 338/670] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Jun 18 04:59:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:09.455432 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6264544-f8b8-406e-aa1e-e39c8d5e17e2 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:09.463767 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85852e33-70e2-45d8-bba9-477ceb9800b5 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:09.490732 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:09.499860 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759a1418-a358-4750-a609-de9e4ee95054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:09.501815 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759a1418-a358-4750-a609-de9e4ee95054 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:09.501815 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759a1418-a358-4750-a609-de9e4ee95054 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:09.501815 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759a1418-a358-4750-a609-de9e4ee95054 None admin] PATH_INFO: `/v3/domains` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:09.502054 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759a1418-a358-4750-a609-de9e4ee95054 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:09.502286 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759a1418-a358-4750-a609-de9e4ee95054 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:09.503655 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759a1418-a358-4750-a609-de9e4ee95054 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:09.510861 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:09.517056 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 333/671] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Jun 18 04:59:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:09.517956 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759a1418-a358-4750-a609-de9e4ee95054 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:09.577240 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6264544-f8b8-406e-aa1e-e39c8d5e17e2 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:09.613734 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fa94ef-1117-4c6c-bcac-621e2829f0e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:09.614527 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fa94ef-1117-4c6c-bcac-621e2829f0e4 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:09.614809 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fa94ef-1117-4c6c-bcac-621e2829f0e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:09.616052 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fa94ef-1117-4c6c-bcac-621e2829f0e4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:09.624420 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759a1418-a358-4750-a609-de9e4ee95054 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:09.626640 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fa94ef-1117-4c6c-bcac-621e2829f0e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 18 04:59:09.635018 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-24fa94ef-1117-4c6c-bcac-621e2829f0e4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 18 04:59:09.635018 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:09.635018 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:59:09.635018 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:59:09.635018 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:59:09.635018 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:59:09.635018 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:59:09.635018 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:59:09.635018 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:59:09.635018 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:59:09.635018 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:59:09.635018 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:59:09.635018 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 18 04:59:09.635018 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 18 04:59:09.635018 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 18 04:59:09.635018 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 18 04:59:09.636065 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 18 04:59:09.636065 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 18 04:59:09.636065 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 18 04:59:09.636065 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 18 04:59:09.636065 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:59:09.636065 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:59:09.636065 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:59:09.636065 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:59:09.636065 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:59:09.636065 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:59:09.636065 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:59:09.636065 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jun 18 04:59:09.636065 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:59:09.636065 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:59:09.636065 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:59:09.636065 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:59:09.636913 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:59:09.636913 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:59:09.636913 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:59:09.636913 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:59:09.636913 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 18 04:59:09.636913 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 18 04:59:09.636913 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 18 04:59:09.636913 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 18 04:59:09.636913 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 18 04:59:09.636913 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 18 04:59:09.636913 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 18 04:59:09.636913 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 04:59:09.636913 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 339/672] 10.222.0.23 () {56 vars in 1102 bytes} [Sat Jun 18 04:59:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:59:09.637623 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fa94ef-1117-4c6c-bcac-621e2829f0e4 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:09.663603 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc40c38-be97-41f8-b355-c57fd9aa5234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:09.664252 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc40c38-be97-41f8-b355-c57fd9aa5234 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:09.664535 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc40c38-be97-41f8-b355-c57fd9aa5234 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:09.664808 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc40c38-be97-41f8-b355-c57fd9aa5234 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:09.665661 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc40c38-be97-41f8-b355-c57fd9aa5234 None admin] RBAC: Adding query filter params (role.id=5ec7b68b793d4a54abd30460961346b9, scope.project.id=c95f8f3084c5472cb8364b7c57207394, user.id=4638e5d1fa124667b07054a87c2d353e) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:09.665977 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc40c38-be97-41f8-b355-c57fd9aa5234 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:09.667411 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc40c38-be97-41f8-b355-c57fd9aa5234 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:09.673142 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 334/673] 10.222.0.23 () {56 vars in 1374 bytes} [Sat Jun 18 04:59:09 2022] GET /identity/v3/role_assignments?user.id=4638e5d1fa124667b07054a87c2d353e&scope.project.id=c95f8f3084c5472cb8364b7c57207394&role.id=5ec7b68b793d4a54abd30460961346b9 => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:09.673934 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc40c38-be97-41f8-b355-c57fd9aa5234 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:09.739756 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:09.751694 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fa94ef-1117-4c6c-bcac-621e2829f0e4 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:09.766109 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:09.791008 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9212e42-a8c1-48f6-96fb-7f470f85b86f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:09.791761 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9212e42-a8c1-48f6-96fb-7f470f85b86f None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:09.792140 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9212e42-a8c1-48f6-96fb-7f470f85b86f None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:09.792319 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9212e42-a8c1-48f6-96fb-7f470f85b86f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:09.799910 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9212e42-a8c1-48f6-96fb-7f470f85b86f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:09.800470 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc40c38-be97-41f8-b355-c57fd9aa5234 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:09.801404 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9212e42-a8c1-48f6-96fb-7f470f85b86f None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:09.818930 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-c9212e42-a8c1-48f6-96fb-7f470f85b86f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 18 04:59:09.818930 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:09.818930 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:59:09.818930 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:59:09.818930 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:59:09.818930 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:59:09.818930 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:59:09.818930 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:59:09.818930 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:59:09.818930 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:59:09.818930 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:59:09.818930 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:59:09.818930 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 18 04:59:09.818930 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 18 04:59:09.818930 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 18 04:59:09.819896 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 18 04:59:09.819896 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:59:09.819896 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:59:09.819896 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:59:09.819896 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:59:09.819896 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:59:09.819896 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:59:09.819896 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:59:09.819896 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jun 18 04:59:09.819896 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:59:09.819896 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:59:09.819896 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:59:09.819896 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:59:09.819896 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:59:09.819896 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:59:09.819896 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:59:09.819896 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:59:09.820775 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 18 04:59:09.820775 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 18 04:59:09.820775 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 18 04:59:09.820775 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 18 04:59:09.820775 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 18 04:59:09.820775 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 18 04:59:09.820775 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 18 04:59:09.820775 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 04:59:09.820775 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 340/674] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Jun 18 04:59:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:59:09.821894 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e8d9cc-02e2-4e11-b4bb-be0925fa6182 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:09.821894 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e8d9cc-02e2-4e11-b4bb-be0925fa6182 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:09.822071 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e8d9cc-02e2-4e11-b4bb-be0925fa6182 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:09.823501 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 341/675] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 04:59:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 04:59:09.826758 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e8d9cc-02e2-4e11-b4bb-be0925fa6182 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:09.850095 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8937af-d903-4aef-928c-09128ccd7f78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:09.850996 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8937af-d903-4aef-928c-09128ccd7f78 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:09.851278 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8937af-d903-4aef-928c-09128ccd7f78 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:09.851547 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8937af-d903-4aef-928c-09128ccd7f78 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:09.861350 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8937af-d903-4aef-928c-09128ccd7f78 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:09.862754 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8937af-d903-4aef-928c-09128ccd7f78 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:09.878840 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application [None req-cd8937af-d903-4aef-928c-09128ccd7f78 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 18 04:59:09.878840 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:09.878840 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:59:09.878840 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:59:09.878840 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:59:09.878840 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:59:09.878840 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:59:09.878840 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:59:09.878840 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:59:09.878840 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:59:09.878840 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:59:09.878840 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:59:09.878840 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 18 04:59:09.878840 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 18 04:59:09.878840 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 18 04:59:09.879773 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 18 04:59:09.879773 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:59:09.879773 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:59:09.879773 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:59:09.879773 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:59:09.879773 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:59:09.879773 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:59:09.879773 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:59:09.879773 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application with Lock( Jun 18 04:59:09.879773 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:59:09.879773 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:59:09.879773 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:59:09.879773 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:59:09.879773 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:59:09.879773 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:59:09.879773 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:59:09.879773 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:59:09.880705 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 18 04:59:09.880705 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 18 04:59:09.880705 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 18 04:59:09.880705 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 18 04:59:09.880705 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 18 04:59:09.880705 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 18 04:59:09.880705 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 18 04:59:09.880705 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 04:59:09.880705 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 335/676] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Jun 18 04:59:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:59:09.881221 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8937af-d903-4aef-928c-09128ccd7f78 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:09.946470 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e8d9cc-02e2-4e11-b4bb-be0925fa6182 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:09.983391 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c866fd9-77e1-4cce-ac1c-1a121a0a571b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:09.984061 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c866fd9-77e1-4cce-ac1c-1a121a0a571b None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:09.984345 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c866fd9-77e1-4cce-ac1c-1a121a0a571b None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:09.984621 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c866fd9-77e1-4cce-ac1c-1a121a0a571b None admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:09.985362 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c866fd9-77e1-4cce-ac1c-1a121a0a571b None admin] RBAC: Adding query filter params (name=service) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:09.985703 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c866fd9-77e1-4cce-ac1c-1a121a0a571b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:09.987079 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c866fd9-77e1-4cce-ac1c-1a121a0a571b None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:09.994717 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8937af-d903-4aef-928c-09128ccd7f78 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:10.000484 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 342/677] 10.222.0.23 () {56 vars in 1103 bytes} [Sat Jun 18 04:59:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:10.001771 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48de669-b813-4a54-8e96-90206696c5b1 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:10.002063 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48de669-b813-4a54-8e96-90206696c5b1 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:10.002347 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48de669-b813-4a54-8e96-90206696c5b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:10.006169 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:10.019189 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-d48de669-b813-4a54-8e96-90206696c5b1 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 04:59:10.020434 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-d48de669-b813-4a54-8e96-90206696c5b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 04:59:10.031995 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3acf09-39c2-457f-8851-d32adffae106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:10.032664 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3acf09-39c2-457f-8851-d32adffae106 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:10.032950 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3acf09-39c2-457f-8851-d32adffae106 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:10.033229 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3acf09-39c2-457f-8851-d32adffae106 None admin] PATH_INFO: `/v3/projects` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:10.034036 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3acf09-39c2-457f-8851-d32adffae106 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:10.034356 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3acf09-39c2-457f-8851-d32adffae106 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:10.035772 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3acf09-39c2-457f-8851-d32adffae106 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:10.047222 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:10.054851 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 336/678] 10.222.0.23 () {56 vars in 1148 bytes} [Sat Jun 18 04:59:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:10.054851 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3acf09-39c2-457f-8851-d32adffae106 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:10.095670 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 343/679] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 04:59:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 18 04:59:10.096677 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48de669-b813-4a54-8e96-90206696c5b1 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:10.176414 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3acf09-39c2-457f-8851-d32adffae106 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:10.211685 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b30ec45-c4be-4c5d-8fa9-1a8c29d4aa20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:10.212564 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48de669-b813-4a54-8e96-90206696c5b1 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:10.212800 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b30ec45-c4be-4c5d-8fa9-1a8c29d4aa20 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:10.213237 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b30ec45-c4be-4c5d-8fa9-1a8c29d4aa20 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:10.213697 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b30ec45-c4be-4c5d-8fa9-1a8c29d4aa20 None admin] PATH_INFO: `/v3/projects` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:10.214622 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b30ec45-c4be-4c5d-8fa9-1a8c29d4aa20 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:10.215081 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b30ec45-c4be-4c5d-8fa9-1a8c29d4aa20 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:10.216551 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b30ec45-c4be-4c5d-8fa9-1a8c29d4aa20 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:10.233545 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 337/680] 10.222.0.23 () {56 vars in 1148 bytes} [Sat Jun 18 04:59:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:10.234388 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b30ec45-c4be-4c5d-8fa9-1a8c29d4aa20 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:10.254477 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0909579f-27f9-49da-9303-4489822eb30c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:10.255193 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0909579f-27f9-49da-9303-4489822eb30c None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:10.255478 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0909579f-27f9-49da-9303-4489822eb30c None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:10.256269 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0909579f-27f9-49da-9303-4489822eb30c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:10.265631 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:10.267862 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0909579f-27f9-49da-9303-4489822eb30c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:10.269337 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0909579f-27f9-49da-9303-4489822eb30c None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:10.288158 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-0909579f-27f9-49da-9303-4489822eb30c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 04:59:10.288158 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:10.288158 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:59:10.288158 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:59:10.288158 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:59:10.288158 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:59:10.288158 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:59:10.288158 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:59:10.288158 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:59:10.288158 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:59:10.288158 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:59:10.288158 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:59:10.288158 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 18 04:59:10.288158 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 18 04:59:10.288158 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 18 04:59:10.295034 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 18 04:59:10.295034 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:59:10.295034 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:59:10.295034 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:59:10.295034 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:59:10.295034 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:59:10.295034 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:59:10.295034 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:59:10.295034 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jun 18 04:59:10.295034 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:59:10.295034 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:59:10.295034 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:59:10.295034 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:59:10.295034 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:59:10.295034 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:59:10.295034 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:59:10.295034 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:59:10.296115 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 18 04:59:10.296115 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 18 04:59:10.296115 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 04:59:10.296115 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 04:59:10.296115 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 344/681] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Jun 18 04:59:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:59:10.296115 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0909579f-27f9-49da-9303-4489822eb30c None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:10.309655 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:10.346089 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b30ec45-c4be-4c5d-8fa9-1a8c29d4aa20 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:10.385490 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd59fa2-a864-4c75-8674-342dcbed31ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:10.386145 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd59fa2-a864-4c75-8674-342dcbed31ea None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:10.386427 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd59fa2-a864-4c75-8674-342dcbed31ea None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:10.386701 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd59fa2-a864-4c75-8674-342dcbed31ea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:10.387653 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd59fa2-a864-4c75-8674-342dcbed31ea None admin] RBAC: Adding query filter params (role.id=5ec7b68b793d4a54abd30460961346b9, scope.project.id=c95f8f3084c5472cb8364b7c57207394, user.id=d9de0428abc8400aa338aba4f7c34c94) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:10.387975 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd59fa2-a864-4c75-8674-342dcbed31ea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:10.389427 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd59fa2-a864-4c75-8674-342dcbed31ea None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:10.398932 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 338/682] 10.222.0.23 () {56 vars in 1374 bytes} [Sat Jun 18 04:59:10 2022] GET /identity/v3/role_assignments?user.id=d9de0428abc8400aa338aba4f7c34c94&scope.project.id=c95f8f3084c5472cb8364b7c57207394&role.id=5ec7b68b793d4a54abd30460961346b9 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:10.399755 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd59fa2-a864-4c75-8674-342dcbed31ea None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:10.417279 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0909579f-27f9-49da-9303-4489822eb30c None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:10.463692 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57d32ad-480a-4ced-be34-bd641d51d339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:10.464775 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57d32ad-480a-4ced-be34-bd641d51d339 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:10.465203 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57d32ad-480a-4ced-be34-bd641d51d339 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:10.465654 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57d32ad-480a-4ced-be34-bd641d51d339 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:10.480277 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57d32ad-480a-4ced-be34-bd641d51d339 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 18 04:59:10.488578 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-b57d32ad-480a-4ced-be34-bd641d51d339 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 18 04:59:10.488578 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:10.488578 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:59:10.488578 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:59:10.488578 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:59:10.488578 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:59:10.488578 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:59:10.488578 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:59:10.488578 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:59:10.488578 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:59:10.488578 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:59:10.488578 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:59:10.488578 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 18 04:59:10.488578 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 18 04:59:10.488578 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 18 04:59:10.488578 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 18 04:59:10.489872 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 18 04:59:10.489872 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 18 04:59:10.489872 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 18 04:59:10.489872 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 18 04:59:10.489872 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:59:10.489872 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:59:10.489872 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:59:10.489872 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:59:10.489872 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:59:10.489872 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:59:10.489872 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:59:10.489872 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jun 18 04:59:10.489872 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:59:10.489872 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:59:10.489872 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:59:10.489872 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:59:10.492950 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:59:10.492950 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:59:10.492950 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:59:10.492950 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:59:10.492950 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 18 04:59:10.492950 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 18 04:59:10.492950 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 18 04:59:10.492950 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 18 04:59:10.492950 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 18 04:59:10.492950 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 18 04:59:10.492950 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 18 04:59:10.492950 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 04:59:10.496143 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 345/683] 10.222.0.23 () {56 vars in 1096 bytes} [Sat Jun 18 04:59:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:59:10.497247 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57d32ad-480a-4ced-be34-bd641d51d339 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:10.526605 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:10.537319 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd59fa2-a864-4c75-8674-342dcbed31ea None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:10.589624 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5156421-24b4-464c-b632-7cadc5b9ff26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:10.590290 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5156421-24b4-464c-b632-7cadc5b9ff26 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:10.590569 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5156421-24b4-464c-b632-7cadc5b9ff26 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:10.590845 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5156421-24b4-464c-b632-7cadc5b9ff26 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:10.592026 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:10.592138 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5156421-24b4-464c-b632-7cadc5b9ff26 None admin] RBAC: Adding query filter params (role.id=5ec7b68b793d4a54abd30460961346b9, scope.project.id=c95f8f3084c5472cb8364b7c57207394, user.id=e78c4d7cde8f4ad9aa479bb09e325db3) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:10.592421 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5156421-24b4-464c-b632-7cadc5b9ff26 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:10.593929 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5156421-24b4-464c-b632-7cadc5b9ff26 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:10.601273 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 339/684] 10.222.0.23 () {56 vars in 1374 bytes} [Sat Jun 18 04:59:10 2022] GET /identity/v3/role_assignments?user.id=e78c4d7cde8f4ad9aa479bb09e325db3&scope.project.id=c95f8f3084c5472cb8364b7c57207394&role.id=5ec7b68b793d4a54abd30460961346b9 => generated 608 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:10.604247 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5156421-24b4-464c-b632-7cadc5b9ff26 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:10.621262 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57d32ad-480a-4ced-be34-bd641d51d339 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:10.671747 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047bea4f-ad17-4700-8ed0-6d0b2aa05378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:10.672583 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047bea4f-ad17-4700-8ed0-6d0b2aa05378 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:10.672993 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047bea4f-ad17-4700-8ed0-6d0b2aa05378 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:10.674119 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047bea4f-ad17-4700-8ed0-6d0b2aa05378 None admin] PATH_INFO: `/v3/domains` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:10.675012 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047bea4f-ad17-4700-8ed0-6d0b2aa05378 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:10.675724 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047bea4f-ad17-4700-8ed0-6d0b2aa05378 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:10.678186 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047bea4f-ad17-4700-8ed0-6d0b2aa05378 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:10.696733 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 346/685] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Jun 18 04:59:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:10.703092 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047bea4f-ad17-4700-8ed0-6d0b2aa05378 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:10.747795 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5156421-24b4-464c-b632-7cadc5b9ff26 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:10.795752 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dde054-7521-4a60-b50c-693d0f17a120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:10.796532 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dde054-7521-4a60-b50c-693d0f17a120 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:10.797653 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dde054-7521-4a60-b50c-693d0f17a120 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:10.797949 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dde054-7521-4a60-b50c-693d0f17a120 None admin] PATH_INFO: `/v3/roles` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:10.798713 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dde054-7521-4a60-b50c-693d0f17a120 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:10.799027 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dde054-7521-4a60-b50c-693d0f17a120 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:10.800391 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dde054-7521-4a60-b50c-693d0f17a120 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:10.815335 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 340/686] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Jun 18 04:59:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:10.825223 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dde054-7521-4a60-b50c-693d0f17a120 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:10.838084 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047bea4f-ad17-4700-8ed0-6d0b2aa05378 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:10.853443 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:10.884222 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d912bec-4021-4b65-81db-d545660d6460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:10.885054 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d912bec-4021-4b65-81db-d545660d6460 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:10.885644 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d912bec-4021-4b65-81db-d545660d6460 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:10.885929 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d912bec-4021-4b65-81db-d545660d6460 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:10.888816 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:10.897693 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d912bec-4021-4b65-81db-d545660d6460 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:10.899656 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d912bec-4021-4b65-81db-d545660d6460 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:10.917785 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-1d912bec-4021-4b65-81db-d545660d6460 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 18 04:59:10.917785 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:10.917785 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:59:10.917785 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:59:10.917785 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:59:10.917785 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:59:10.917785 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:59:10.917785 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:59:10.917785 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:59:10.917785 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:59:10.917785 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:59:10.917785 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:59:10.917785 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 18 04:59:10.917785 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 18 04:59:10.917785 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 18 04:59:10.917785 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 18 04:59:10.918791 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:59:10.918791 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:59:10.918791 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 18 04:59:10.918791 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 18 04:59:10.918791 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 18 04:59:10.918791 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 18 04:59:10.918791 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:59:10.918791 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:59:10.918791 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:59:10.918791 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:59:10.918791 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:59:10.918791 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jun 18 04:59:10.918791 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:59:10.918791 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:59:10.918791 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:59:10.918791 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:59:10.918791 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:59:10.918791 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:59:10.919757 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:59:10.919757 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:59:10.919757 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 18 04:59:10.919757 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 18 04:59:10.919757 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 18 04:59:10.919757 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 18 04:59:10.919757 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 18 04:59:10.919757 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 18 04:59:10.919757 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 18 04:59:10.919757 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 04:59:10.922421 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 347/687] 10.222.0.23 () {56 vars in 1096 bytes} [Sat Jun 18 04:59:10 2022] GET /identity/v3/users/swift => generated 83 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 18 04:59:10.930039 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d912bec-4021-4b65-81db-d545660d6460 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:10.952609 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dde054-7521-4a60-b50c-693d0f17a120 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:10.996132 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4355bf47-9f7d-4c0e-915b-dde4d305f96e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:10.997355 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4355bf47-9f7d-4c0e-915b-dde4d305f96e None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:10.997866 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4355bf47-9f7d-4c0e-915b-dde4d305f96e None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:10.998354 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4355bf47-9f7d-4c0e-915b-dde4d305f96e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:11.012283 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4355bf47-9f7d-4c0e-915b-dde4d305f96e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:11.014200 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4355bf47-9f7d-4c0e-915b-dde4d305f96e None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:11.032685 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application [None req-4355bf47-9f7d-4c0e-915b-dde4d305f96e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 04:59:11.032685 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:11.032685 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:59:11.032685 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:59:11.032685 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:59:11.032685 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:59:11.032685 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:59:11.032685 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:59:11.032685 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:59:11.032685 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:59:11.032685 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:59:11.032685 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:59:11.032685 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 18 04:59:11.032685 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 18 04:59:11.032685 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 18 04:59:11.033636 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 18 04:59:11.033636 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:59:11.033636 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:59:11.033636 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:59:11.033636 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:59:11.033636 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:59:11.033636 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:59:11.033636 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:59:11.033636 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application with Lock( Jun 18 04:59:11.033636 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:59:11.033636 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:59:11.033636 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:59:11.033636 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:59:11.033636 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:59:11.033636 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:59:11.033636 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:59:11.033636 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:59:11.034519 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 18 04:59:11.034519 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 18 04:59:11.034519 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 04:59:11.034519 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 04:59:11.037253 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 341/688] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Jun 18 04:59:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:59:11.043180 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4355bf47-9f7d-4c0e-915b-dde4d305f96e None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:11.045918 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d912bec-4021-4b65-81db-d545660d6460 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:11.091316 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6aa74ed-6844-4700-b97b-764903caecfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:11.092692 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6aa74ed-6844-4700-b97b-764903caecfd None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:11.092692 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6aa74ed-6844-4700-b97b-764903caecfd None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:11.092968 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6aa74ed-6844-4700-b97b-764903caecfd None admin] PATH_INFO: `/v3/users` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:11.093812 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6aa74ed-6844-4700-b97b-764903caecfd None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:11.094123 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6aa74ed-6844-4700-b97b-764903caecfd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:11.096154 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6aa74ed-6844-4700-b97b-764903caecfd None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:11.124408 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 348/689] 10.222.0.23 () {56 vars in 1135 bytes} [Sat Jun 18 04:59:10 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:11.125800 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6aa74ed-6844-4700-b97b-764903caecfd None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:11.139386 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:11.203327 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4355bf47-9f7d-4c0e-915b-dde4d305f96e None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:11.229748 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:11.280130 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6aa74ed-6844-4700-b97b-764903caecfd None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:11.317945 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdb2dfe-3fd7-4edd-b017-c16d10b6b1f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:11.318664 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdb2dfe-3fd7-4edd-b017-c16d10b6b1f7 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:11.318948 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdb2dfe-3fd7-4edd-b017-c16d10b6b1f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:11.319221 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdb2dfe-3fd7-4edd-b017-c16d10b6b1f7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:11.342105 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9cf1a1-d4fa-41b2-a5ff-8f60074c3756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:11.342799 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9cf1a1-d4fa-41b2-a5ff-8f60074c3756 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:11.343094 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9cf1a1-d4fa-41b2-a5ff-8f60074c3756 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:11.343377 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9cf1a1-d4fa-41b2-a5ff-8f60074c3756 None admin] PATH_INFO: `/v3/domains` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:11.344112 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9cf1a1-d4fa-41b2-a5ff-8f60074c3756 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:11.344426 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9cf1a1-d4fa-41b2-a5ff-8f60074c3756 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:11.345843 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9cf1a1-d4fa-41b2-a5ff-8f60074c3756 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:11.384321 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 342/690] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Jun 18 04:59:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 18 04:59:11.388714 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9cf1a1-d4fa-41b2-a5ff-8f60074c3756 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:11.404036 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdb2dfe-3fd7-4edd-b017-c16d10b6b1f7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:11.405534 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdb2dfe-3fd7-4edd-b017-c16d10b6b1f7 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:11.513610 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:11.517720 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9cf1a1-d4fa-41b2-a5ff-8f60074c3756 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:11.554069 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:11.559512 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18717fb-a174-47bc-8629-410b6a4ee60b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:11.560304 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18717fb-a174-47bc-8629-410b6a4ee60b None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:11.560583 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18717fb-a174-47bc-8629-410b6a4ee60b None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:11.560858 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18717fb-a174-47bc-8629-410b6a4ee60b None admin] PATH_INFO: `/v3/users/nova` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:11.572881 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18717fb-a174-47bc-8629-410b6a4ee60b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:11.572881 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18717fb-a174-47bc-8629-410b6a4ee60b None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:11.589281 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application [None req-d18717fb-a174-47bc-8629-410b6a4ee60b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 18 04:59:11.589281 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:11.589281 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:59:11.589281 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:59:11.589281 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:59:11.589281 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:59:11.589281 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:59:11.589281 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:59:11.589281 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:59:11.589281 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:59:11.589281 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:59:11.589281 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:59:11.589281 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 18 04:59:11.589281 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 18 04:59:11.589281 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 18 04:59:11.589281 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 18 04:59:11.590267 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:59:11.590267 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:59:11.590267 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 18 04:59:11.590267 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 18 04:59:11.590267 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 18 04:59:11.590267 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 18 04:59:11.590267 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:59:11.590267 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:59:11.590267 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:59:11.590267 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:59:11.590267 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:59:11.590267 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application with Lock( Jun 18 04:59:11.590267 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:59:11.590267 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:59:11.590267 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:59:11.590267 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:59:11.590267 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:59:11.590267 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:59:11.591276 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:59:11.591276 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:59:11.591276 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 18 04:59:11.591276 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 18 04:59:11.591276 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 18 04:59:11.591276 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 18 04:59:11.591276 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 18 04:59:11.591276 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 18 04:59:11.591276 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 18 04:59:11.591276 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 04:59:11.591276 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 343/691] 10.222.0.23 () {56 vars in 1093 bytes} [Sat Jun 18 04:59:11 2022] GET /identity/v3/users/nova => generated 82 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 18 04:59:11.597610 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18717fb-a174-47bc-8629-410b6a4ee60b None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:11.626126 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 349/692] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Jun 18 04:59:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 501 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 18 04:59:11.632787 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdb2dfe-3fd7-4edd-b017-c16d10b6b1f7 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:11.708160 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18717fb-a174-47bc-8629-410b6a4ee60b None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:11.746618 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c0f6c9-13c1-4121-9186-b73d49541e61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:11.747286 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c0f6c9-13c1-4121-9186-b73d49541e61 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:11.747568 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c0f6c9-13c1-4121-9186-b73d49541e61 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:11.747841 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c0f6c9-13c1-4121-9186-b73d49541e61 None admin] PATH_INFO: `/v3/users` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:11.748794 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c0f6c9-13c1-4121-9186-b73d49541e61 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:11.749319 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c0f6c9-13c1-4121-9186-b73d49541e61 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:11.750839 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c0f6c9-13c1-4121-9186-b73d49541e61 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:11.771988 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 344/693] 10.222.0.23 () {56 vars in 1133 bytes} [Sat Jun 18 04:59:11 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:11.777126 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c0f6c9-13c1-4121-9186-b73d49541e61 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:11.823096 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdb2dfe-3fd7-4edd-b017-c16d10b6b1f7 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:11.874905 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:11.921228 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855385d9-2858-4bd4-b46f-5885d0287d41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:11.922056 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855385d9-2858-4bd4-b46f-5885d0287d41 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:11.922343 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855385d9-2858-4bd4-b46f-5885d0287d41 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:11.922625 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855385d9-2858-4bd4-b46f-5885d0287d41 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:11.930896 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c0f6c9-13c1-4121-9186-b73d49541e61 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:11.933357 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855385d9-2858-4bd4-b46f-5885d0287d41 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:11.934862 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855385d9-2858-4bd4-b46f-5885d0287d41 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:11.954016 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-855385d9-2858-4bd4-b46f-5885d0287d41 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 04:59:11.954016 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:11.954016 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:59:11.954016 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:59:11.954016 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:59:11.954016 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:59:11.954016 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:59:11.954016 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:59:11.954016 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:59:11.954016 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:59:11.954016 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:59:11.954016 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:59:11.954016 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 18 04:59:11.954016 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 18 04:59:11.954016 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 18 04:59:11.954917 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 18 04:59:11.954917 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:59:11.954917 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:59:11.954917 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:59:11.954917 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:59:11.954917 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:59:11.954917 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:59:11.954917 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:59:11.954917 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jun 18 04:59:11.954917 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:59:11.954917 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:59:11.954917 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:59:11.954917 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:59:11.954917 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:59:11.954917 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:59:11.954917 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:59:11.954917 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:59:11.955784 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 18 04:59:11.955784 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 18 04:59:11.955784 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 04:59:11.955784 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 04:59:11.955784 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 350/694] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Jun 18 04:59:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:59:11.956894 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61da68e-74df-49f8-975b-ad0ba00f46c2 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:11.957183 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61da68e-74df-49f8-975b-ad0ba00f46c2 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:11.957484 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61da68e-74df-49f8-975b-ad0ba00f46c2 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:11.963295 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 351/695] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 04:59:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 04:59:11.964165 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61da68e-74df-49f8-975b-ad0ba00f46c2 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:11.967163 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e2ff0f-b105-465d-9de0-d36a92579921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:11.967805 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:11.968488 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e2ff0f-b105-465d-9de0-d36a92579921 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:11.968775 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e2ff0f-b105-465d-9de0-d36a92579921 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:11.969136 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e2ff0f-b105-465d-9de0-d36a92579921 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:12.043421 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e2ff0f-b105-465d-9de0-d36a92579921 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:12.044987 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e2ff0f-b105-465d-9de0-d36a92579921 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:12.087662 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61da68e-74df-49f8-975b-ad0ba00f46c2 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:12.099141 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:12.128625 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10731b6-3d44-4642-8454-1066fb7b320f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:12.129287 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10731b6-3d44-4642-8454-1066fb7b320f None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:12.129599 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10731b6-3d44-4642-8454-1066fb7b320f None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:12.129878 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10731b6-3d44-4642-8454-1066fb7b320f None admin] PATH_INFO: `/v3/domains` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:12.130607 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10731b6-3d44-4642-8454-1066fb7b320f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:12.130916 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10731b6-3d44-4642-8454-1066fb7b320f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:12.132784 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10731b6-3d44-4642-8454-1066fb7b320f None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:12.147313 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 352/696] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Jun 18 04:59:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:12.148994 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4120f0c-5f40-41f1-b36c-1120004f17e4 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:12.149713 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4120f0c-5f40-41f1-b36c-1120004f17e4 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:12.150001 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4120f0c-5f40-41f1-b36c-1120004f17e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:12.160328 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 345/697] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Jun 18 04:59:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 18 04:59:12.166525 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e2ff0f-b105-465d-9de0-d36a92579921 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:12.168976 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-b4120f0c-5f40-41f1-b36c-1120004f17e4 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 04:59:12.171384 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-b4120f0c-5f40-41f1-b36c-1120004f17e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 04:59:12.249890 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:12.259171 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 353/698] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 04:59:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 18 04:59:12.260272 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4120f0c-5f40-41f1-b36c-1120004f17e4 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:12.289981 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e2ff0f-b105-465d-9de0-d36a92579921 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:12.338816 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ceda507-d710-41e4-a2e9-abb91c562c4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:12.339578 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceda507-d710-41e4-a2e9-abb91c562c4c None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:12.339868 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceda507-d710-41e4-a2e9-abb91c562c4c None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:12.340146 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceda507-d710-41e4-a2e9-abb91c562c4c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:12.350399 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ceda507-d710-41e4-a2e9-abb91c562c4c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:12.352164 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ceda507-d710-41e4-a2e9-abb91c562c4c None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:12.370758 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4120f0c-5f40-41f1-b36c-1120004f17e4 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:12.371230 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application [None req-4ceda507-d710-41e4-a2e9-abb91c562c4c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 18 04:59:12.371230 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:12.371230 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:59:12.371230 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:59:12.371230 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:59:12.371230 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:59:12.371230 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:59:12.371230 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:59:12.371230 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:59:12.371230 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:59:12.371230 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:59:12.371230 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:59:12.371230 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 18 04:59:12.371230 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 18 04:59:12.371230 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 18 04:59:12.372063 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 18 04:59:12.372063 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:59:12.372063 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:59:12.372063 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:59:12.372063 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:59:12.372063 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:59:12.372063 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:59:12.372063 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:59:12.372063 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application with Lock( Jun 18 04:59:12.372063 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:59:12.372063 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:59:12.372063 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:59:12.372063 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:59:12.372063 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:59:12.372063 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:59:12.372063 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:59:12.372063 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:59:12.372959 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 18 04:59:12.372959 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 18 04:59:12.372959 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 18 04:59:12.372959 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 18 04:59:12.372959 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 18 04:59:12.372959 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 18 04:59:12.372959 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 18 04:59:12.372959 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 04:59:12.372959 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 346/699] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Jun 18 04:59:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:59:12.378570 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ceda507-d710-41e4-a2e9-abb91c562c4c None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:12.430287 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3df3ea-4fde-466f-a18f-c648fba96fe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:12.431716 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3df3ea-4fde-466f-a18f-c648fba96fe4 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:12.432415 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3df3ea-4fde-466f-a18f-c648fba96fe4 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:12.432926 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3df3ea-4fde-466f-a18f-c648fba96fe4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:12.433732 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:12.447260 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3df3ea-4fde-466f-a18f-c648fba96fe4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:12.449810 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3df3ea-4fde-466f-a18f-c648fba96fe4 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:12.473497 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-7c3df3ea-4fde-466f-a18f-c648fba96fe4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 04:59:12.473497 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:12.473497 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:59:12.473497 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:59:12.473497 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:59:12.473497 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:59:12.473497 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:59:12.473497 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:59:12.473497 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:59:12.473497 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:59:12.473497 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:59:12.473497 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:59:12.473497 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 18 04:59:12.473497 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 18 04:59:12.473497 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 18 04:59:12.474951 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 18 04:59:12.474951 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:59:12.474951 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:59:12.474951 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:59:12.474951 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:59:12.474951 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:59:12.474951 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:59:12.474951 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:59:12.474951 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jun 18 04:59:12.474951 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:59:12.474951 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:59:12.474951 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:59:12.474951 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:59:12.474951 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:59:12.474951 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:59:12.474951 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:59:12.474951 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:59:12.476003 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 18 04:59:12.476003 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 18 04:59:12.476003 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 04:59:12.476003 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 04:59:12.482662 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 354/700] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Jun 18 04:59:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:59:12.483775 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3df3ea-4fde-466f-a18f-c648fba96fe4 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:12.503013 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ceda507-d710-41e4-a2e9-abb91c562c4c None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:12.536388 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:12.555230 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b22dc58-f64f-4dad-b765-3d5cf9077b4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:12.556223 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b22dc58-f64f-4dad-b765-3d5cf9077b4a None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:12.556675 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b22dc58-f64f-4dad-b765-3d5cf9077b4a None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:12.557096 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b22dc58-f64f-4dad-b765-3d5cf9077b4a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:12.569311 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b22dc58-f64f-4dad-b765-3d5cf9077b4a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:12.571013 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b22dc58-f64f-4dad-b765-3d5cf9077b4a None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:12.588719 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application [None req-3b22dc58-f64f-4dad-b765-3d5cf9077b4a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 04:59:12.588719 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:12.588719 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:59:12.588719 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:59:12.588719 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:59:12.588719 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:59:12.588719 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:59:12.588719 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:59:12.588719 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:59:12.588719 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:59:12.588719 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:59:12.588719 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:59:12.588719 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 18 04:59:12.588719 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 18 04:59:12.588719 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 18 04:59:12.589865 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 18 04:59:12.589865 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:59:12.589865 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:59:12.589865 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:59:12.589865 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:59:12.589865 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:59:12.589865 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:59:12.589865 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:59:12.589865 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application with Lock( Jun 18 04:59:12.589865 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:59:12.589865 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:59:12.589865 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:59:12.589865 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:59:12.589865 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:59:12.589865 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:59:12.589865 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:59:12.589865 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:59:12.591047 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 18 04:59:12.591047 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 18 04:59:12.591047 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 04:59:12.591047 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 04:59:12.593115 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 347/701] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Jun 18 04:59:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:59:12.594130 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b22dc58-f64f-4dad-b765-3d5cf9077b4a None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:12.608868 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3df3ea-4fde-466f-a18f-c648fba96fe4 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:12.653661 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4254f7bc-02c1-4b20-99a9-4a912ab94dcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:12.654467 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4254f7bc-02c1-4b20-99a9-4a912ab94dcb None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:12.655176 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4254f7bc-02c1-4b20-99a9-4a912ab94dcb None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:12.655559 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4254f7bc-02c1-4b20-99a9-4a912ab94dcb None admin] PATH_INFO: `/v3/projects` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:12.656734 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4254f7bc-02c1-4b20-99a9-4a912ab94dcb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:12.657146 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4254f7bc-02c1-4b20-99a9-4a912ab94dcb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:12.659517 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4254f7bc-02c1-4b20-99a9-4a912ab94dcb None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:12.660833 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:12.681123 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 355/702] 10.222.0.23 () {56 vars in 1148 bytes} [Sat Jun 18 04:59:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:12.681123 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4254f7bc-02c1-4b20-99a9-4a912ab94dcb None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:12.712068 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b22dc58-f64f-4dad-b765-3d5cf9077b4a None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:12.750542 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2f0adf-9d23-4c53-a655-b51e814ace23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:12.751580 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2f0adf-9d23-4c53-a655-b51e814ace23 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:12.752022 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2f0adf-9d23-4c53-a655-b51e814ace23 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:12.752430 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2f0adf-9d23-4c53-a655-b51e814ace23 None admin] PATH_INFO: `/v3/domains` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:12.753290 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2f0adf-9d23-4c53-a655-b51e814ace23 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:12.753775 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2f0adf-9d23-4c53-a655-b51e814ace23 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:12.755419 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2f0adf-9d23-4c53-a655-b51e814ace23 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:12.774604 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 348/703] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Jun 18 04:59:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:12.775799 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2f0adf-9d23-4c53-a655-b51e814ace23 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:12.793437 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4254f7bc-02c1-4b20-99a9-4a912ab94dcb None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:12.800814 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:12.830002 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58409cd2-d826-4300-9cd9-d6832fd3d5ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:12.830669 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58409cd2-d826-4300-9cd9-d6832fd3d5ac None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:12.830954 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58409cd2-d826-4300-9cd9-d6832fd3d5ac None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:12.831231 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58409cd2-d826-4300-9cd9-d6832fd3d5ac None admin] PATH_INFO: `/v3/domains` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:12.832023 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58409cd2-d826-4300-9cd9-d6832fd3d5ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:12.832531 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58409cd2-d826-4300-9cd9-d6832fd3d5ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:12.835171 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58409cd2-d826-4300-9cd9-d6832fd3d5ac None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:12.849943 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 356/704] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Jun 18 04:59:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:12.849943 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58409cd2-d826-4300-9cd9-d6832fd3d5ac None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:12.886489 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2f0adf-9d23-4c53-a655-b51e814ace23 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:12.911016 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:12.923315 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78207ecc-27a6-43f5-8ae3-0d072bf3c71f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:12.923976 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78207ecc-27a6-43f5-8ae3-0d072bf3c71f None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:12.924254 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78207ecc-27a6-43f5-8ae3-0d072bf3c71f None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:12.924695 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78207ecc-27a6-43f5-8ae3-0d072bf3c71f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:12.925415 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78207ecc-27a6-43f5-8ae3-0d072bf3c71f None admin] RBAC: Adding query filter params (role.id=5ec7b68b793d4a54abd30460961346b9, scope.project.id=c95f8f3084c5472cb8364b7c57207394, user.id=16b49e866398400580ae48aa5a96fc04) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:12.925720 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78207ecc-27a6-43f5-8ae3-0d072bf3c71f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:12.927147 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78207ecc-27a6-43f5-8ae3-0d072bf3c71f None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:12.939815 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 349/705] 10.222.0.23 () {56 vars in 1374 bytes} [Sat Jun 18 04:59:12 2022] GET /identity/v3/role_assignments?user.id=16b49e866398400580ae48aa5a96fc04&scope.project.id=c95f8f3084c5472cb8364b7c57207394&role.id=5ec7b68b793d4a54abd30460961346b9 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:12.939815 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f110ef1-652f-4eef-8e8b-180b17c3dc57 None None] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:12.939815 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f110ef1-652f-4eef-8e8b-180b17c3dc57 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:12.939815 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f110ef1-652f-4eef-8e8b-180b17c3dc57 None None] PATH_INFO: `/` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:12.939815 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 350/706] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 04:59:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 04:59:12.939815 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f110ef1-652f-4eef-8e8b-180b17c3dc57 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:12.984865 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58409cd2-d826-4300-9cd9-d6832fd3d5ac None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:13.031685 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac911655-02cf-4b79-971d-da1e183b9fcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:13.032521 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac911655-02cf-4b79-971d-da1e183b9fcf None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:13.032810 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac911655-02cf-4b79-971d-da1e183b9fcf None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:13.033087 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac911655-02cf-4b79-971d-da1e183b9fcf None admin] PATH_INFO: `/v3/projects/service` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:13.041799 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac911655-02cf-4b79-971d-da1e183b9fcf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:13.043274 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac911655-02cf-4b79-971d-da1e183b9fcf None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:13.061498 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-ac911655-02cf-4b79-971d-da1e183b9fcf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 18 04:59:13.061498 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:13.061498 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:59:13.061498 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:59:13.061498 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:59:13.061498 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:59:13.061498 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:59:13.061498 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:59:13.061498 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:59:13.061498 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:59:13.061498 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:59:13.061498 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:59:13.061498 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 18 04:59:13.061498 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 18 04:59:13.061498 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 18 04:59:13.062477 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 18 04:59:13.062477 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:59:13.062477 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:59:13.062477 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:59:13.062477 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:59:13.062477 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:59:13.062477 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:59:13.062477 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:59:13.062477 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jun 18 04:59:13.062477 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:59:13.062477 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:59:13.062477 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:59:13.062477 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:59:13.062477 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:59:13.062477 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:59:13.062477 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:59:13.062477 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:59:13.063390 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 18 04:59:13.063390 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 18 04:59:13.063390 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 18 04:59:13.063390 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 18 04:59:13.063390 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 18 04:59:13.063390 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 18 04:59:13.063390 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 18 04:59:13.063390 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 04:59:13.063390 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 357/707] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Jun 18 04:59:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:59:13.064421 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e86b711-3add-46cc-bae0-392cf7832c09 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:13.066483 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e86b711-3add-46cc-bae0-392cf7832c09 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:13.066483 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e86b711-3add-46cc-bae0-392cf7832c09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:13.083206 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f110ef1-652f-4eef-8e8b-180b17c3dc57 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:13.083446 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-7e86b711-3add-46cc-bae0-392cf7832c09 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 04:59:13.085344 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-7e86b711-3add-46cc-bae0-392cf7832c09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 04:59:13.109361 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:13.137998 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a31bb7-3632-4b63-98a2-916809b67ab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:13.139039 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a31bb7-3632-4b63-98a2-916809b67ab8 None admin] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:13.139555 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a31bb7-3632-4b63-98a2-916809b67ab8 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:13.140086 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a31bb7-3632-4b63-98a2-916809b67ab8 None admin] PATH_INFO: `/v3/users` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:13.141001 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a31bb7-3632-4b63-98a2-916809b67ab8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:13.142751 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a31bb7-3632-4b63-98a2-916809b67ab8 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:13.170677 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 358/708] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 04:59:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 18 04:59:13.171352 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e86b711-3add-46cc-bae0-392cf7832c09 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:13.218069 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 351/709] 10.222.0.23 () {60 vars in 1132 bytes} [Sat Jun 18 04:59:12 2022] POST /identity/v3/users => generated 254 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 04:59:13.219260 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a31bb7-3632-4b63-98a2-916809b67ab8 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:13.296450 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e86b711-3add-46cc-bae0-392cf7832c09 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:13.326192 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:13.342030 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165a516e-0915-4bd0-a4be-2c24368704e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:13.342812 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165a516e-0915-4bd0-a4be-2c24368704e8 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:13.343097 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165a516e-0915-4bd0-a4be-2c24368704e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:13.343370 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165a516e-0915-4bd0-a4be-2c24368704e8 None admin] PATH_INFO: `/v3/projects` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:13.344899 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165a516e-0915-4bd0-a4be-2c24368704e8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:13.345215 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165a516e-0915-4bd0-a4be-2c24368704e8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:13.346964 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165a516e-0915-4bd0-a4be-2c24368704e8 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:13.361960 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 359/710] 10.222.0.23 () {56 vars in 1148 bytes} [Sat Jun 18 04:59:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:13.363522 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ceee429-6212-4df2-95eb-a2cea266e184 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:13.363823 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ceee429-6212-4df2-95eb-a2cea266e184 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:13.364098 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ceee429-6212-4df2-95eb-a2cea266e184 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:13.366756 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 360/711] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 04:59:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 04:59:13.371781 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ceee429-6212-4df2-95eb-a2cea266e184 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:13.397902 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a31bb7-3632-4b63-98a2-916809b67ab8 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:13.417079 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:13.442128 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cb29b8-f85e-4463-8b10-f81618bb329e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:13.443927 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cb29b8-f85e-4463-8b10-f81618bb329e None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:13.444487 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cb29b8-f85e-4463-8b10-f81618bb329e None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:13.444928 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cb29b8-f85e-4463-8b10-f81618bb329e None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:13.446507 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cb29b8-f85e-4463-8b10-f81618bb329e None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:13.448483 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cb29b8-f85e-4463-8b10-f81618bb329e None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:13.465236 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application [None req-44cb29b8-f85e-4463-8b10-f81618bb329e None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 18 04:59:13.465236 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:13.465236 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 18 04:59:13.465236 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 18 04:59:13.465236 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 18 04:59:13.465236 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 18 04:59:13.465236 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 18 04:59:13.465236 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 18 04:59:13.465236 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 18 04:59:13.465236 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 04:59:13.465236 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 18 04:59:13.465236 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 04:59:13.465236 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:13.465236 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:59:13.465236 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:59:13.465236 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:59:13.466514 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:59:13.466514 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:59:13.466514 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:59:13.466514 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:59:13.466514 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:59:13.466514 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:59:13.466514 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:59:13.466514 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 18 04:59:13.466514 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 18 04:59:13.466514 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 18 04:59:13.466514 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 18 04:59:13.466514 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:59:13.466514 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:59:13.466514 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:59:13.466514 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:59:13.466514 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:59:13.467564 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:59:13.467564 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:59:13.467564 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application with Lock( Jun 18 04:59:13.467564 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:59:13.467564 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:59:13.467564 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:59:13.467564 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:59:13.467564 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:59:13.467564 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:59:13.467564 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:59:13.467564 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:59:13.467564 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 18 04:59:13.467564 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 18 04:59:13.467564 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 18 04:59:13.467564 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 04:59:13.469156 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 352/712] 10.222.0.23 () {56 vars in 1129 bytes} [Sat Jun 18 04:59:13 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:59:13.470306 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cf1674-9a53-495b-a3aa-153fa1304ac4 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:13.470592 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cf1674-9a53-495b-a3aa-153fa1304ac4 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:13.470864 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cf1674-9a53-495b-a3aa-153fa1304ac4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:13.496293 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-c5cf1674-9a53-495b-a3aa-153fa1304ac4 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 04:59:13.498870 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-c5cf1674-9a53-495b-a3aa-153fa1304ac4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 04:59:13.513225 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ceee429-6212-4df2-95eb-a2cea266e184 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:13.558494 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e259d94-5ca0-483d-9a97-d6f80bfc53dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:13.559247 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e259d94-5ca0-483d-9a97-d6f80bfc53dd None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:13.559540 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e259d94-5ca0-483d-9a97-d6f80bfc53dd None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:13.559814 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e259d94-5ca0-483d-9a97-d6f80bfc53dd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:13.560703 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e259d94-5ca0-483d-9a97-d6f80bfc53dd None admin] RBAC: Adding query filter params (role.id=21c65f2c51bd4dd283d13507d2714467, scope.project.id=c95f8f3084c5472cb8364b7c57207394, user.id=15e7f45ee0744c1d9602187eac8e8b80) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:13.560950 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e259d94-5ca0-483d-9a97-d6f80bfc53dd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:13.564294 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e259d94-5ca0-483d-9a97-d6f80bfc53dd None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:13.569610 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 361/713] 10.222.0.23 () {56 vars in 1374 bytes} [Sat Jun 18 04:59:13 2022] GET /identity/v3/role_assignments?user.id=15e7f45ee0744c1d9602187eac8e8b80&scope.project.id=c95f8f3084c5472cb8364b7c57207394&role.id=21c65f2c51bd4dd283d13507d2714467 => generated 259 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:13.570522 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e259d94-5ca0-483d-9a97-d6f80bfc53dd None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:13.593663 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 353/714] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 04:59:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 18 04:59:13.619434 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cf1674-9a53-495b-a3aa-153fa1304ac4 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:13.682130 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:13.682328 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:13.685206 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e259d94-5ca0-483d-9a97-d6f80bfc53dd None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:13.732823 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06cfd2a-6d3a-4a10-8b13-b9b5bfdee112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:13.733544 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06cfd2a-6d3a-4a10-8b13-b9b5bfdee112 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:13.733829 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06cfd2a-6d3a-4a10-8b13-b9b5bfdee112 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:13.734104 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06cfd2a-6d3a-4a10-8b13-b9b5bfdee112 None admin] PATH_INFO: `/v3/services` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:13.734822 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06cfd2a-6d3a-4a10-8b13-b9b5bfdee112 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:13.735134 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06cfd2a-6d3a-4a10-8b13-b9b5bfdee112 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:13.736550 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06cfd2a-6d3a-4a10-8b13-b9b5bfdee112 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:13.745246 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 362/715] 10.222.0.23 () {56 vars in 1124 bytes} [Sat Jun 18 04:59:13 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:13.754336 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06cfd2a-6d3a-4a10-8b13-b9b5bfdee112 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:13.805848 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cf1674-9a53-495b-a3aa-153fa1304ac4 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:13.870802 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d8d0c8-5eda-46ce-a804-40b8307cdf7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:13.871520 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d8d0c8-5eda-46ce-a804-40b8307cdf7e None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:13.871804 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d8d0c8-5eda-46ce-a804-40b8307cdf7e None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:13.872078 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d8d0c8-5eda-46ce-a804-40b8307cdf7e None admin] PATH_INFO: `/v3/services/network` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:13.872738 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d8d0c8-5eda-46ce-a804-40b8307cdf7e None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:13.874131 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d8d0c8-5eda-46ce-a804-40b8307cdf7e None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:13.906264 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application [None req-53d8d0c8-5eda-46ce-a804-40b8307cdf7e None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 18 04:59:13.906264 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:13.906264 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 18 04:59:13.906264 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 18 04:59:13.906264 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 18 04:59:13.906264 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 18 04:59:13.906264 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 18 04:59:13.906264 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 18 04:59:13.906264 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 18 04:59:13.906264 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 04:59:13.906264 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 18 04:59:13.906264 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 04:59:13.906264 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:13.906264 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:59:13.906264 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:59:13.906264 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:59:13.906264 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:59:13.907740 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:59:13.907740 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:59:13.907740 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:59:13.907740 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:59:13.907740 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:59:13.907740 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:59:13.907740 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 18 04:59:13.907740 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 18 04:59:13.907740 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 18 04:59:13.907740 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 18 04:59:13.907740 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:59:13.907740 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:59:13.907740 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:59:13.907740 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:59:13.907740 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:59:13.907740 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:59:13.909251 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:59:13.909251 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application with Lock( Jun 18 04:59:13.909251 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:59:13.909251 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:59:13.909251 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:59:13.909251 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:59:13.909251 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:59:13.909251 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:59:13.909251 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:59:13.909251 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:59:13.909251 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 18 04:59:13.909251 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 18 04:59:13.909251 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 18 04:59:13.909251 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 04:59:13.909251 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 354/716] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Jun 18 04:59:13 2022] GET /identity/v3/services/network => generated 88 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 18 04:59:13.914564 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06cfd2a-6d3a-4a10-8b13-b9b5bfdee112 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:13.914812 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d8d0c8-5eda-46ce-a804-40b8307cdf7e None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:13.954311 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bbfbb8-ff64-4d97-96c4-c0f3ee4998b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:13.954982 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bbfbb8-ff64-4d97-96c4-c0f3ee4998b8 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:13.955265 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bbfbb8-ff64-4d97-96c4-c0f3ee4998b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:13.956503 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bbfbb8-ff64-4d97-96c4-c0f3ee4998b8 None admin] PATH_INFO: `/v3/services` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:13.957265 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bbfbb8-ff64-4d97-96c4-c0f3ee4998b8 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:13.957604 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bbfbb8-ff64-4d97-96c4-c0f3ee4998b8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:13.962571 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bbfbb8-ff64-4d97-96c4-c0f3ee4998b8 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:13.971990 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 363/717] 10.222.0.23 () {56 vars in 1124 bytes} [Sat Jun 18 04:59:13 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:14.094030 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:14.097717 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d8d0c8-5eda-46ce-a804-40b8307cdf7e None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:14.141608 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e5f27f-8cdd-40c7-a7c2-14dc59f8f842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:14.142197 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e5f27f-8cdd-40c7-a7c2-14dc59f8f842 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:14.142478 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e5f27f-8cdd-40c7-a7c2-14dc59f8f842 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:14.142748 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e5f27f-8cdd-40c7-a7c2-14dc59f8f842 None admin] PATH_INFO: `/v3/services` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:14.143454 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e5f27f-8cdd-40c7-a7c2-14dc59f8f842 None admin] RBAC: Adding query filter params (name=network) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:14.143758 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e5f27f-8cdd-40c7-a7c2-14dc59f8f842 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:14.145125 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e5f27f-8cdd-40c7-a7c2-14dc59f8f842 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:14.151781 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 355/718] 10.222.0.23 () {56 vars in 1112 bytes} [Sat Jun 18 04:59:13 2022] GET /identity/v3/services?name=network => generated 124 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:14.157340 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bbfbb8-ff64-4d97-96c4-c0f3ee4998b8 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:14.179796 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:14.287976 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bbfbb8-ff64-4d97-96c4-c0f3ee4998b8 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:14.336705 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6e9fe6-21aa-40fe-8cc1-f3aba404efdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:14.337376 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6e9fe6-21aa-40fe-8cc1-f3aba404efdc None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:14.337682 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6e9fe6-21aa-40fe-8cc1-f3aba404efdc None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:14.337961 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6e9fe6-21aa-40fe-8cc1-f3aba404efdc None admin] PATH_INFO: `/v3/services` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:14.338670 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6e9fe6-21aa-40fe-8cc1-f3aba404efdc None admin] RBAC: Adding query filter params (type=network) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:14.338989 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6e9fe6-21aa-40fe-8cc1-f3aba404efdc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:14.340378 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6e9fe6-21aa-40fe-8cc1-f3aba404efdc None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:14.349919 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 364/719] 10.222.0.23 () {56 vars in 1112 bytes} [Sat Jun 18 04:59:14 2022] GET /identity/v3/services?type=network => generated 124 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:15.562721 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f36db8c-2c2b-4142-8127-91822e149456 None None] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:15.563225 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f36db8c-2c2b-4142-8127-91822e149456 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:15.563299 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f36db8c-2c2b-4142-8127-91822e149456 None None] PATH_INFO: `/` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:15.564863 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 356/720] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 04:59:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 04:59:15.570290 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d179bb52-340c-4ada-9369-0146f14d710a None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:15.570748 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d179bb52-340c-4ada-9369-0146f14d710a None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:15.570829 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d179bb52-340c-4ada-9369-0146f14d710a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:15.594357 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-d179bb52-340c-4ada-9369-0146f14d710a None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 04:59:15.595557 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-d179bb52-340c-4ada-9369-0146f14d710a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 04:59:15.685118 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 365/721] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 04:59:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 18 04:59:15.732367 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f36db8c-2c2b-4142-8127-91822e149456 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:15.882113 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f36db8c-2c2b-4142-8127-91822e149456 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:15.931982 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b090210-3952-4fc2-9580-d387507bdb91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:15.932782 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b090210-3952-4fc2-9580-d387507bdb91 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:15.933063 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b090210-3952-4fc2-9580-d387507bdb91 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:15.933337 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b090210-3952-4fc2-9580-d387507bdb91 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:15.950287 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b090210-3952-4fc2-9580-d387507bdb91 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 18 04:59:15.957682 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application [None req-3b090210-3952-4fc2-9580-d387507bdb91 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 18 04:59:15.957682 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:15.957682 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:59:15.957682 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:59:15.957682 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:59:15.957682 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:59:15.957682 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:59:15.957682 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:59:15.957682 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:59:15.957682 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:59:15.957682 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:59:15.957682 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:59:15.957682 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 18 04:59:15.957682 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 18 04:59:15.957682 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 18 04:59:15.957682 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 18 04:59:15.959138 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 18 04:59:15.959138 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 18 04:59:15.959138 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 18 04:59:15.959138 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 18 04:59:15.959138 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:59:15.959138 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:59:15.959138 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:59:15.959138 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:59:15.959138 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:59:15.959138 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:59:15.959138 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:59:15.959138 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application with Lock( Jun 18 04:59:15.959138 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:59:15.959138 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:59:15.959138 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:59:15.959138 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:59:15.959971 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:59:15.959971 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:59:15.959971 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:59:15.959971 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:59:15.959971 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 18 04:59:15.959971 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 18 04:59:15.959971 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 18 04:59:15.959971 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 18 04:59:15.959971 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 18 04:59:15.959971 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 18 04:59:15.959971 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 18 04:59:15.959971 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 04:59:15.959971 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 357/722] 10.222.0.23 () {56 vars in 1096 bytes} [Sat Jun 18 04:59:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:59:15.965971 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d179bb52-340c-4ada-9369-0146f14d710a None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:15.968104 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:15.977662 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5ee067-a221-4a73-8424-9261410f550d None None] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:15.978528 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5ee067-a221-4a73-8424-9261410f550d None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:15.978528 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5ee067-a221-4a73-8424-9261410f550d None None] PATH_INFO: `/` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:15.979820 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 358/723] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 04:59:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 04:59:15.984579 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7fe836-29c8-4997-8e38-c07f8c943708 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:15.984865 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7fe836-29c8-4997-8e38-c07f8c943708 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:15.985142 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7fe836-29c8-4997-8e38-c07f8c943708 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:16.018407 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-8f7fe836-29c8-4997-8e38-c07f8c943708 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 04:59:16.019596 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-8f7fe836-29c8-4997-8e38-c07f8c943708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 04:59:16.061365 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:16.084599 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d179bb52-340c-4ada-9369-0146f14d710a None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:16.133213 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de0b66a-f087-4380-b958-a5a65d63d636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:16.133909 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de0b66a-f087-4380-b958-a5a65d63d636 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:16.134189 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de0b66a-f087-4380-b958-a5a65d63d636 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:16.134464 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de0b66a-f087-4380-b958-a5a65d63d636 None admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:16.135205 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de0b66a-f087-4380-b958-a5a65d63d636 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:16.135514 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de0b66a-f087-4380-b958-a5a65d63d636 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:16.136888 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de0b66a-f087-4380-b958-a5a65d63d636 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:16.158737 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 366/724] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Jun 18 04:59:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:16.164434 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de0b66a-f087-4380-b958-a5a65d63d636 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:16.184723 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 359/725] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 04:59:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 18 04:59:16.219313 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7fe836-29c8-4997-8e38-c07f8c943708 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:16.261943 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:16.304040 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de0b66a-f087-4380-b958-a5a65d63d636 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:16.333404 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7fe836-29c8-4997-8e38-c07f8c943708 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:16.340835 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951df467-7bd9-4627-a3e7-dc1f87d0d07e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:16.341861 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951df467-7bd9-4627-a3e7-dc1f87d0d07e None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:16.342295 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951df467-7bd9-4627-a3e7-dc1f87d0d07e None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:16.342719 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951df467-7bd9-4627-a3e7-dc1f87d0d07e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:16.354265 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951df467-7bd9-4627-a3e7-dc1f87d0d07e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:16.355912 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951df467-7bd9-4627-a3e7-dc1f87d0d07e None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:16.372425 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4039640-2df8-451d-8460-c6a15718c4fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:16.373400 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4039640-2df8-451d-8460-c6a15718c4fa None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:16.373790 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4039640-2df8-451d-8460-c6a15718c4fa None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:16.374169 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4039640-2df8-451d-8460-c6a15718c4fa None admin] PATH_INFO: `/v3/roles/service` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:16.375164 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-951df467-7bd9-4627-a3e7-dc1f87d0d07e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 04:59:16.375164 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:16.375164 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:59:16.375164 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:59:16.375164 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:59:16.375164 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:59:16.375164 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:59:16.375164 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:59:16.375164 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:59:16.375164 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:59:16.375164 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:59:16.375164 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:59:16.375164 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 18 04:59:16.375164 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 18 04:59:16.375164 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 18 04:59:16.376169 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 18 04:59:16.376169 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:59:16.376169 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:59:16.376169 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:59:16.376169 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:59:16.376169 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:59:16.376169 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:59:16.376169 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:59:16.376169 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jun 18 04:59:16.376169 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:59:16.376169 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:59:16.376169 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:59:16.376169 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:59:16.376169 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:59:16.376169 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:59:16.376169 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:59:16.376169 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:59:16.377089 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 18 04:59:16.377089 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 18 04:59:16.377089 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 04:59:16.377089 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 04:59:16.379339 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 367/726] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Jun 18 04:59:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:59:16.381659 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a9a894-db69-416e-b790-9313d5c1ab61 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:16.382181 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a9a894-db69-416e-b790-9313d5c1ab61 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:16.382626 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a9a894-db69-416e-b790-9313d5c1ab61 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:16.384345 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 368/727] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 04:59:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 04:59:16.385613 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a9a894-db69-416e-b790-9313d5c1ab61 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:16.401062 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4039640-2df8-451d-8460-c6a15718c4fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 18 04:59:16.409456 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application [None req-b4039640-2df8-451d-8460-c6a15718c4fa None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 18 04:59:16.409456 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:16.409456 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:59:16.409456 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:59:16.409456 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:59:16.409456 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:59:16.409456 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:59:16.409456 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:59:16.409456 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:59:16.409456 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:59:16.409456 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:59:16.409456 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:59:16.409456 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 18 04:59:16.409456 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 18 04:59:16.409456 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 18 04:59:16.409456 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 18 04:59:16.410421 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 18 04:59:16.410421 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 18 04:59:16.410421 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 18 04:59:16.410421 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 18 04:59:16.410421 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:59:16.410421 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:59:16.410421 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:59:16.410421 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:59:16.410421 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:59:16.410421 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:59:16.410421 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:59:16.410421 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application with Lock( Jun 18 04:59:16.410421 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:59:16.410421 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:59:16.410421 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:59:16.410421 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:59:16.412359 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:59:16.412359 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:59:16.412359 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:59:16.412359 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:59:16.412359 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 18 04:59:16.412359 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 18 04:59:16.412359 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 18 04:59:16.412359 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 18 04:59:16.412359 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 18 04:59:16.412359 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 18 04:59:16.412359 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 18 04:59:16.412359 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 04:59:16.412359 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 360/728] 10.222.0.23 () {56 vars in 1102 bytes} [Sat Jun 18 04:59:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 18 04:59:16.413198 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b38616-125f-4bce-ac52-fe8948ed2639 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:16.413694 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b38616-125f-4bce-ac52-fe8948ed2639 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:16.414115 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b38616-125f-4bce-ac52-fe8948ed2639 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:16.432254 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-88b38616-125f-4bce-ac52-fe8948ed2639 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 04:59:16.433723 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-88b38616-125f-4bce-ac52-fe8948ed2639 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 04:59:16.444056 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:16.503118 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a9a894-db69-416e-b790-9313d5c1ab61 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:16.513682 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 361/729] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 04:59:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 18 04:59:16.514834 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b38616-125f-4bce-ac52-fe8948ed2639 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:16.537820 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:16.542182 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d63e21-1285-4028-b373-679ea738b7e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:16.543023 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d63e21-1285-4028-b373-679ea738b7e5 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:16.543415 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d63e21-1285-4028-b373-679ea738b7e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:16.543796 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d63e21-1285-4028-b373-679ea738b7e5 None admin] PATH_INFO: `/v3/domains` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:16.544642 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d63e21-1285-4028-b373-679ea738b7e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:16.545054 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d63e21-1285-4028-b373-679ea738b7e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:16.546610 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d63e21-1285-4028-b373-679ea738b7e5 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:16.560021 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 369/730] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Jun 18 04:59:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:16.561620 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99d643f-0dea-4de3-9ba7-fc237ddf0d9d None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:16.561970 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99d643f-0dea-4de3-9ba7-fc237ddf0d9d None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:16.562874 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99d643f-0dea-4de3-9ba7-fc237ddf0d9d None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:16.563902 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 370/731] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 04:59:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 04:59:16.564779 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99d643f-0dea-4de3-9ba7-fc237ddf0d9d None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:16.633529 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b38616-125f-4bce-ac52-fe8948ed2639 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:16.669753 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068fa899-13bd-4ec3-9e58-e6d508cdbc38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:16.670581 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068fa899-13bd-4ec3-9e58-e6d508cdbc38 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:16.671166 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068fa899-13bd-4ec3-9e58-e6d508cdbc38 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:16.671453 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068fa899-13bd-4ec3-9e58-e6d508cdbc38 None admin] PATH_INFO: `/v3/roles` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:16.672365 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068fa899-13bd-4ec3-9e58-e6d508cdbc38 None admin] RBAC: Adding query filter params (name=service) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:16.672676 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068fa899-13bd-4ec3-9e58-e6d508cdbc38 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:16.674064 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068fa899-13bd-4ec3-9e58-e6d508cdbc38 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:16.674610 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99d643f-0dea-4de3-9ba7-fc237ddf0d9d None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:16.687686 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 362/732] 10.222.0.23 () {56 vars in 1103 bytes} [Sat Jun 18 04:59:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:16.687686 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068fa899-13bd-4ec3-9e58-e6d508cdbc38 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:16.695644 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:16.718515 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5181c652-3e7e-4939-9416-a29c0ea56d27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:16.719305 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5181c652-3e7e-4939-9416-a29c0ea56d27 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:16.719590 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5181c652-3e7e-4939-9416-a29c0ea56d27 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:16.719907 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5181c652-3e7e-4939-9416-a29c0ea56d27 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:16.730480 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5181c652-3e7e-4939-9416-a29c0ea56d27 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 18 04:59:16.738259 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-5181c652-3e7e-4939-9416-a29c0ea56d27 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 18 04:59:16.738259 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:16.738259 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:59:16.738259 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:59:16.738259 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:59:16.738259 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:59:16.738259 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:59:16.738259 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:59:16.738259 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:59:16.738259 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:59:16.738259 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:59:16.738259 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:59:16.738259 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 18 04:59:16.738259 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 18 04:59:16.738259 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 18 04:59:16.738259 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 18 04:59:16.743418 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 18 04:59:16.743418 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 18 04:59:16.743418 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 18 04:59:16.743418 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 18 04:59:16.743418 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:59:16.743418 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:59:16.743418 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:59:16.743418 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:59:16.743418 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:59:16.743418 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:59:16.743418 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:59:16.743418 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jun 18 04:59:16.743418 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:59:16.743418 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:59:16.743418 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:59:16.743418 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:59:16.744265 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:59:16.744265 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:59:16.744265 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:59:16.744265 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:59:16.744265 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 18 04:59:16.744265 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 18 04:59:16.744265 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 18 04:59:16.744265 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 18 04:59:16.744265 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 18 04:59:16.744265 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 18 04:59:16.744265 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 18 04:59:16.744265 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 04:59:16.744265 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 371/733] 10.222.0.23 () {56 vars in 1096 bytes} [Sat Jun 18 04:59:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:59:16.744265 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c9dc2b-dcf0-4ef0-9cc0-00de6a0d094c None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:16.745034 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c9dc2b-dcf0-4ef0-9cc0-00de6a0d094c None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:16.745034 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c9dc2b-dcf0-4ef0-9cc0-00de6a0d094c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:16.772407 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-72c9dc2b-dcf0-4ef0-9cc0-00de6a0d094c None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 04:59:16.773567 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-72c9dc2b-dcf0-4ef0-9cc0-00de6a0d094c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 04:59:16.804990 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:16.830117 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068fa899-13bd-4ec3-9e58-e6d508cdbc38 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:16.857664 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 372/734] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 04:59:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 18 04:59:16.858394 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c9dc2b-dcf0-4ef0-9cc0-00de6a0d094c None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:16.867996 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac922221-fe9b-4ae7-8ca0-c18257cefd63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:16.868957 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac922221-fe9b-4ae7-8ca0-c18257cefd63 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:16.869213 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac922221-fe9b-4ae7-8ca0-c18257cefd63 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:16.869526 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac922221-fe9b-4ae7-8ca0-c18257cefd63 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:16.878586 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac922221-fe9b-4ae7-8ca0-c18257cefd63 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:16.880243 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac922221-fe9b-4ae7-8ca0-c18257cefd63 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:16.896430 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application [None req-ac922221-fe9b-4ae7-8ca0-c18257cefd63 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 18 04:59:16.896430 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:16.896430 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:59:16.896430 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:59:16.896430 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:59:16.896430 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:59:16.896430 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:59:16.896430 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:59:16.896430 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:59:16.896430 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:59:16.896430 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:59:16.896430 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:59:16.896430 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 18 04:59:16.896430 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 18 04:59:16.896430 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 18 04:59:16.896430 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 18 04:59:16.897442 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:59:16.897442 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:59:16.897442 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 18 04:59:16.897442 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 18 04:59:16.897442 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 18 04:59:16.897442 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 18 04:59:16.897442 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:59:16.897442 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:59:16.897442 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:59:16.897442 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:59:16.897442 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:59:16.897442 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application with Lock( Jun 18 04:59:16.897442 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:59:16.897442 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:59:16.897442 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:59:16.897442 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:59:16.897442 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:59:16.897442 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:59:16.898387 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:59:16.898387 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:59:16.898387 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 18 04:59:16.898387 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 18 04:59:16.898387 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 18 04:59:16.898387 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 18 04:59:16.898387 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 18 04:59:16.898387 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 18 04:59:16.898387 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 18 04:59:16.898387 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 04:59:16.898387 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 363/735] 10.222.0.23 () {56 vars in 1096 bytes} [Sat Jun 18 04:59:16 2022] GET /identity/v3/users/swift => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:59:16.898974 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac922221-fe9b-4ae7-8ca0-c18257cefd63 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:16.967231 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c9dc2b-dcf0-4ef0-9cc0-00de6a0d094c None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:16.973265 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:17.000160 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6039f6b-d53a-4543-96f7-31dcf6690b6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:17.000940 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6039f6b-d53a-4543-96f7-31dcf6690b6f None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:17.001222 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6039f6b-d53a-4543-96f7-31dcf6690b6f None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:17.001527 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6039f6b-d53a-4543-96f7-31dcf6690b6f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:17.008918 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac922221-fe9b-4ae7-8ca0-c18257cefd63 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:17.012096 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6039f6b-d53a-4543-96f7-31dcf6690b6f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:17.013593 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6039f6b-d53a-4543-96f7-31dcf6690b6f None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:17.031683 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-a6039f6b-d53a-4543-96f7-31dcf6690b6f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 04:59:17.031683 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:17.031683 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:59:17.031683 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:59:17.031683 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:59:17.031683 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:59:17.031683 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:59:17.031683 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:59:17.031683 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:59:17.031683 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:59:17.031683 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:59:17.031683 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:59:17.031683 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 18 04:59:17.031683 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 18 04:59:17.031683 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 18 04:59:17.032715 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 18 04:59:17.032715 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:59:17.032715 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:59:17.032715 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:59:17.032715 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:59:17.032715 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:59:17.032715 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:59:17.032715 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:59:17.032715 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jun 18 04:59:17.032715 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:59:17.032715 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:59:17.032715 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:59:17.032715 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:59:17.032715 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:59:17.032715 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:59:17.032715 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:59:17.032715 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:59:17.033640 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 18 04:59:17.033640 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 18 04:59:17.033640 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 04:59:17.033640 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 04:59:17.033640 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 373/736] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Jun 18 04:59:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:59:17.033640 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6039f6b-d53a-4543-96f7-31dcf6690b6f None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:17.045136 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509e7ff7-6bac-4027-97f1-8bf8fc1c5fdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:17.045849 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509e7ff7-6bac-4027-97f1-8bf8fc1c5fdf None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:17.046130 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509e7ff7-6bac-4027-97f1-8bf8fc1c5fdf None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:17.046404 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509e7ff7-6bac-4027-97f1-8bf8fc1c5fdf None admin] PATH_INFO: `/v3/roles` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:17.047143 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509e7ff7-6bac-4027-97f1-8bf8fc1c5fdf None admin] RBAC: Adding query filter params (name=admin) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:17.047458 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509e7ff7-6bac-4027-97f1-8bf8fc1c5fdf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:17.048801 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509e7ff7-6bac-4027-97f1-8bf8fc1c5fdf None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:17.055405 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:17.056803 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 364/737] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Jun 18 04:59:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:17.057605 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509e7ff7-6bac-4027-97f1-8bf8fc1c5fdf None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:17.136010 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6039f6b-d53a-4543-96f7-31dcf6690b6f None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:17.160509 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509e7ff7-6bac-4027-97f1-8bf8fc1c5fdf None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:17.169925 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139e8a02-7be1-42e0-9994-d1523d7f2940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:17.170585 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139e8a02-7be1-42e0-9994-d1523d7f2940 None admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:17.170869 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139e8a02-7be1-42e0-9994-d1523d7f2940 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:17.171148 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139e8a02-7be1-42e0-9994-d1523d7f2940 None admin] PATH_INFO: `/v3/services` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:17.171873 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139e8a02-7be1-42e0-9994-d1523d7f2940 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:17.173244 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139e8a02-7be1-42e0-9994-d1523d7f2940 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:17.187468 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:17.195124 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71051327-c2a6-46a6-8b5f-cf9f152ed115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:17.195786 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71051327-c2a6-46a6-8b5f-cf9f152ed115 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:17.203701 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71051327-c2a6-46a6-8b5f-cf9f152ed115 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:17.203701 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71051327-c2a6-46a6-8b5f-cf9f152ed115 None admin] PATH_INFO: `/v3/users` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:17.203701 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71051327-c2a6-46a6-8b5f-cf9f152ed115 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:17.203701 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71051327-c2a6-46a6-8b5f-cf9f152ed115 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:17.203701 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71051327-c2a6-46a6-8b5f-cf9f152ed115 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:17.216609 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 365/738] 10.222.0.23 () {56 vars in 1135 bytes} [Sat Jun 18 04:59:17 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:17.217993 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5828b88-2649-4c0b-8948-660627af1fc9 None None] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:17.218287 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5828b88-2649-4c0b-8948-660627af1fc9 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:17.218564 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5828b88-2649-4c0b-8948-660627af1fc9 None None] PATH_INFO: `/` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:17.219795 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 366/739] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 04:59:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 04:59:17.220485 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5828b88-2649-4c0b-8948-660627af1fc9 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:17.282512 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 374/740] 10.222.0.23 () {60 vars in 1141 bytes} [Sat Jun 18 04:59:17 2022] POST /identity/v3/services => generated 251 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 04:59:17.283233 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139e8a02-7be1-42e0-9994-d1523d7f2940 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:17.351566 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5828b88-2649-4c0b-8948-660627af1fc9 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:17.380111 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:17.398329 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6babf130-9b09-4c44-9e58-cd1ae1d6274a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:17.399010 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6babf130-9b09-4c44-9e58-cd1ae1d6274a None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:17.399292 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6babf130-9b09-4c44-9e58-cd1ae1d6274a None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:17.399569 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6babf130-9b09-4c44-9e58-cd1ae1d6274a None admin] PATH_INFO: `/v3/domains` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:17.400306 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6babf130-9b09-4c44-9e58-cd1ae1d6274a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:17.400614 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6babf130-9b09-4c44-9e58-cd1ae1d6274a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:17.402060 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6babf130-9b09-4c44-9e58-cd1ae1d6274a None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:17.407525 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139e8a02-7be1-42e0-9994-d1523d7f2940 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:17.415650 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 367/741] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Jun 18 04:59:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:17.416330 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6babf130-9b09-4c44-9e58-cd1ae1d6274a None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:17.459318 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44898c1-9936-4640-8cfd-c4f9f4632bd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:17.460123 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44898c1-9936-4640-8cfd-c4f9f4632bd9 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:17.460399 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44898c1-9936-4640-8cfd-c4f9f4632bd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:17.460676 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44898c1-9936-4640-8cfd-c4f9f4632bd9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:17.467330 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:17.472581 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44898c1-9936-4640-8cfd-c4f9f4632bd9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:17.474090 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44898c1-9936-4640-8cfd-c4f9f4632bd9 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:17.492785 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-a44898c1-9936-4640-8cfd-c4f9f4632bd9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 04:59:17.492785 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:17.492785 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:59:17.492785 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:59:17.492785 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:59:17.492785 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:59:17.492785 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:59:17.492785 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:59:17.492785 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:59:17.492785 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:59:17.492785 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:59:17.492785 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:59:17.492785 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 18 04:59:17.492785 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 18 04:59:17.492785 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 18 04:59:17.493837 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 18 04:59:17.493837 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:59:17.493837 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:59:17.493837 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:59:17.493837 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:59:17.493837 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:59:17.493837 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:59:17.493837 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:59:17.493837 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jun 18 04:59:17.493837 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:59:17.493837 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:59:17.493837 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:59:17.493837 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:59:17.493837 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:59:17.493837 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:59:17.493837 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:59:17.493837 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:59:17.494733 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 18 04:59:17.494733 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 18 04:59:17.494733 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 04:59:17.494733 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 04:59:17.494733 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 375/742] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Jun 18 04:59:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:59:17.495731 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addf2173-b9df-4dbb-87c9-dd4458cb7a57 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:17.496020 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addf2173-b9df-4dbb-87c9-dd4458cb7a57 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:17.496297 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addf2173-b9df-4dbb-87c9-dd4458cb7a57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:17.513931 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-addf2173-b9df-4dbb-87c9-dd4458cb7a57 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 04:59:17.515144 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-addf2173-b9df-4dbb-87c9-dd4458cb7a57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 04:59:17.539316 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6babf130-9b09-4c44-9e58-cd1ae1d6274a None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:17.575997 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da939cde-50b4-44a6-bcff-1833606a4ea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:17.576658 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da939cde-50b4-44a6-bcff-1833606a4ea9 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:17.576985 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da939cde-50b4-44a6-bcff-1833606a4ea9 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:17.577253 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da939cde-50b4-44a6-bcff-1833606a4ea9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:17.587292 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 376/743] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 04:59:17 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 18 04:59:17.595787 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addf2173-b9df-4dbb-87c9-dd4458cb7a57 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:17.650955 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da939cde-50b4-44a6-bcff-1833606a4ea9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:17.652400 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da939cde-50b4-44a6-bcff-1833606a4ea9 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:17.673536 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:17.698020 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:17.718876 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addf2173-b9df-4dbb-87c9-dd4458cb7a57 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:17.757223 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fe119f-23ea-4375-9423-22b82de7f849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:17.758077 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fe119f-23ea-4375-9423-22b82de7f849 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:17.758356 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fe119f-23ea-4375-9423-22b82de7f849 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:17.758627 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fe119f-23ea-4375-9423-22b82de7f849 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:17.764633 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 368/744] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Jun 18 04:59:17 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 18 04:59:17.765966 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da939cde-50b4-44a6-bcff-1833606a4ea9 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:17.767884 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fe119f-23ea-4375-9423-22b82de7f849 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:17.769711 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fe119f-23ea-4375-9423-22b82de7f849 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:17.786466 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-d5fe119f-23ea-4375-9423-22b82de7f849 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 18 04:59:17.786466 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:17.786466 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:59:17.786466 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:59:17.786466 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:59:17.786466 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:59:17.786466 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:59:17.786466 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:59:17.786466 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:59:17.786466 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:59:17.786466 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:59:17.786466 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:59:17.786466 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 18 04:59:17.786466 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 18 04:59:17.786466 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 18 04:59:17.786466 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 18 04:59:17.788069 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:59:17.788069 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:59:17.788069 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 18 04:59:17.788069 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 18 04:59:17.788069 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 18 04:59:17.788069 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 18 04:59:17.788069 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:59:17.788069 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:59:17.788069 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:59:17.788069 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:59:17.788069 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:59:17.788069 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jun 18 04:59:17.788069 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:59:17.788069 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:59:17.788069 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:59:17.788069 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:59:17.788069 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:59:17.788069 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:59:17.789040 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:59:17.789040 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:59:17.789040 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 18 04:59:17.789040 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 18 04:59:17.789040 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 18 04:59:17.789040 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 18 04:59:17.789040 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 18 04:59:17.789040 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 18 04:59:17.789040 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 18 04:59:17.789040 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 04:59:17.789040 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 377/745] 10.222.0.23 () {56 vars in 1117 bytes} [Sat Jun 18 04:59:17 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:59:17.790665 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fe119f-23ea-4375-9423-22b82de7f849 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:17.873153 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da939cde-50b4-44a6-bcff-1833606a4ea9 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:17.897315 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fe119f-23ea-4375-9423-22b82de7f849 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:17.910470 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba1d194-bc3b-4ee8-9c92-fa151e0447b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:17.911240 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba1d194-bc3b-4ee8-9c92-fa151e0447b1 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:17.911524 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba1d194-bc3b-4ee8-9c92-fa151e0447b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:17.912293 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba1d194-bc3b-4ee8-9c92-fa151e0447b1 None admin] PATH_INFO: `/v3/domains` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:17.913004 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba1d194-bc3b-4ee8-9c92-fa151e0447b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:17.913315 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba1d194-bc3b-4ee8-9c92-fa151e0447b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:17.914931 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba1d194-bc3b-4ee8-9c92-fa151e0447b1 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:17.917550 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:17.929560 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 369/746] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Jun 18 04:59:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:17.930451 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba1d194-bc3b-4ee8-9c92-fa151e0447b1 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:17.934504 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:17.940112 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5badd2ad-7a76-4cce-b096-93a8444e945a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:17.940808 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5badd2ad-7a76-4cce-b096-93a8444e945a None admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:17.941091 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5badd2ad-7a76-4cce-b096-93a8444e945a None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:17.941365 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5badd2ad-7a76-4cce-b096-93a8444e945a None admin] PATH_INFO: `/v3/services` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:17.942114 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5badd2ad-7a76-4cce-b096-93a8444e945a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:17.943589 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5badd2ad-7a76-4cce-b096-93a8444e945a None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:18.031609 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba1d194-bc3b-4ee8-9c92-fa151e0447b1 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:18.054023 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 378/747] 10.222.0.23 () {60 vars in 1141 bytes} [Sat Jun 18 04:59:17 2022] POST /identity/v3/services => generated 240 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 04:59:18.054023 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5badd2ad-7a76-4cce-b096-93a8444e945a None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:18.095506 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46156568-2d91-46e8-8932-afabea70d665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:18.096712 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46156568-2d91-46e8-8932-afabea70d665 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:18.096986 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46156568-2d91-46e8-8932-afabea70d665 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:18.097581 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46156568-2d91-46e8-8932-afabea70d665 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:18.112557 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46156568-2d91-46e8-8932-afabea70d665 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:18.114901 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46156568-2d91-46e8-8932-afabea70d665 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:18.139683 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application [None req-46156568-2d91-46e8-8932-afabea70d665 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 04:59:18.139683 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:18.139683 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:59:18.139683 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:59:18.139683 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:59:18.139683 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:59:18.139683 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:59:18.139683 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:59:18.139683 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:59:18.139683 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:59:18.139683 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:59:18.139683 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:59:18.139683 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 18 04:59:18.139683 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 18 04:59:18.139683 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 18 04:59:18.140828 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 18 04:59:18.140828 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:59:18.140828 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:59:18.140828 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:59:18.140828 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:59:18.140828 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:59:18.140828 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:59:18.140828 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:59:18.140828 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application with Lock( Jun 18 04:59:18.140828 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:59:18.140828 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:59:18.140828 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:59:18.140828 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:59:18.140828 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:59:18.140828 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:59:18.140828 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:59:18.140828 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:59:18.141938 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 18 04:59:18.141938 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 18 04:59:18.141938 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 04:59:18.141938 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 04:59:18.141938 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 370/748] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Jun 18 04:59:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:59:18.142679 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46156568-2d91-46e8-8932-afabea70d665 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:18.168939 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5badd2ad-7a76-4cce-b096-93a8444e945a None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:18.206804 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f04e02d-5ff5-4727-883d-f8a590d67979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:18.207646 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f04e02d-5ff5-4727-883d-f8a590d67979 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:18.208009 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f04e02d-5ff5-4727-883d-f8a590d67979 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:18.208353 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f04e02d-5ff5-4727-883d-f8a590d67979 None admin] PATH_INFO: `/v3/users` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:18.210559 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f04e02d-5ff5-4727-883d-f8a590d67979 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:18.210943 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f04e02d-5ff5-4727-883d-f8a590d67979 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:18.212476 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f04e02d-5ff5-4727-883d-f8a590d67979 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:18.217366 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:18.231915 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 379/749] 10.222.0.23 () {56 vars in 1149 bytes} [Sat Jun 18 04:59:18 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:18.232769 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f04e02d-5ff5-4727-883d-f8a590d67979 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:18.281573 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:18.290871 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46156568-2d91-46e8-8932-afabea70d665 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:18.336459 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b642a239-c65f-4386-accc-c8c187e69463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:18.337547 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b642a239-c65f-4386-accc-c8c187e69463 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:18.337845 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b642a239-c65f-4386-accc-c8c187e69463 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:18.338179 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b642a239-c65f-4386-accc-c8c187e69463 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:18.347666 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f04e02d-5ff5-4727-883d-f8a590d67979 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:18.348670 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b642a239-c65f-4386-accc-c8c187e69463 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:18.351494 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b642a239-c65f-4386-accc-c8c187e69463 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:18.370292 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application [None req-b642a239-c65f-4386-accc-c8c187e69463 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 18 04:59:18.370292 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:18.370292 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:59:18.370292 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:59:18.370292 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:59:18.370292 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:59:18.370292 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:59:18.370292 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:59:18.370292 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:59:18.370292 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:59:18.370292 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:59:18.370292 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:59:18.370292 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 18 04:59:18.370292 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 18 04:59:18.370292 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 18 04:59:18.370292 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 18 04:59:18.371286 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:59:18.371286 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:59:18.371286 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 18 04:59:18.371286 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 18 04:59:18.371286 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 18 04:59:18.371286 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 18 04:59:18.371286 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:59:18.371286 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:59:18.371286 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:59:18.371286 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:59:18.371286 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:59:18.371286 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application with Lock( Jun 18 04:59:18.371286 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:59:18.371286 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:59:18.371286 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:59:18.371286 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:59:18.371286 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:59:18.371286 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:59:18.372237 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:59:18.372237 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:59:18.372237 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 18 04:59:18.372237 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 18 04:59:18.372237 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 18 04:59:18.372237 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 18 04:59:18.372237 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 18 04:59:18.372237 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 18 04:59:18.372237 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 18 04:59:18.372237 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 04:59:18.372237 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 371/750] 10.222.0.23 () {56 vars in 1093 bytes} [Sat Jun 18 04:59:18 2022] GET /identity/v3/users/nova => generated 82 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:59:18.373427 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b642a239-c65f-4386-accc-c8c187e69463 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:18.399079 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56eaa6d0-fcfd-4b52-91b3-a15098590665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:18.400107 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eaa6d0-fcfd-4b52-91b3-a15098590665 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:18.400625 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eaa6d0-fcfd-4b52-91b3-a15098590665 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:18.401209 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eaa6d0-fcfd-4b52-91b3-a15098590665 None admin] PATH_INFO: `/v3/domains` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:18.402521 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56eaa6d0-fcfd-4b52-91b3-a15098590665 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:18.403091 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56eaa6d0-fcfd-4b52-91b3-a15098590665 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:18.404745 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56eaa6d0-fcfd-4b52-91b3-a15098590665 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:18.419125 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 380/751] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Jun 18 04:59:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:18.420216 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56eaa6d0-fcfd-4b52-91b3-a15098590665 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:18.514673 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b642a239-c65f-4386-accc-c8c187e69463 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:18.528487 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:18.528789 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:18.532959 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56eaa6d0-fcfd-4b52-91b3-a15098590665 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:18.561698 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f99928-07a3-4c8f-a81b-0a66e74c0930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:18.562863 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f99928-07a3-4c8f-a81b-0a66e74c0930 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:18.563298 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f99928-07a3-4c8f-a81b-0a66e74c0930 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:18.563717 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f99928-07a3-4c8f-a81b-0a66e74c0930 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:18.573954 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fcff48-2ce2-41c0-a210-b60a2bd3ead4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:18.574789 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fcff48-2ce2-41c0-a210-b60a2bd3ead4 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:18.575273 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fcff48-2ce2-41c0-a210-b60a2bd3ead4 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:18.575907 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fcff48-2ce2-41c0-a210-b60a2bd3ead4 None admin] PATH_INFO: `/v3/users` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:18.576640 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fcff48-2ce2-41c0-a210-b60a2bd3ead4 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:18.577048 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fcff48-2ce2-41c0-a210-b60a2bd3ead4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:18.578606 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fcff48-2ce2-41c0-a210-b60a2bd3ead4 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:18.605145 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 381/752] 10.222.0.23 () {56 vars in 1133 bytes} [Sat Jun 18 04:59:18 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:18.605145 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fcff48-2ce2-41c0-a210-b60a2bd3ead4 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:18.650883 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f99928-07a3-4c8f-a81b-0a66e74c0930 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:18.652327 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f99928-07a3-4c8f-a81b-0a66e74c0930 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:18.745334 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fcff48-2ce2-41c0-a210-b60a2bd3ead4 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:18.773424 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 372/753] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Jun 18 04:59:18 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 18 04:59:18.774510 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f99928-07a3-4c8f-a81b-0a66e74c0930 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:18.787426 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84467496-fb90-4f4d-86c4-783fceaf6e4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:18.788303 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84467496-fb90-4f4d-86c4-783fceaf6e4d None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:18.788657 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84467496-fb90-4f4d-86c4-783fceaf6e4d None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:18.789003 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84467496-fb90-4f4d-86c4-783fceaf6e4d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:18.790795 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:18.796302 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84467496-fb90-4f4d-86c4-783fceaf6e4d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:18.798066 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84467496-fb90-4f4d-86c4-783fceaf6e4d None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:18.812848 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-84467496-fb90-4f4d-86c4-783fceaf6e4d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 18 04:59:18.812848 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:18.812848 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:59:18.812848 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:59:18.812848 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:59:18.812848 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:59:18.812848 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:59:18.812848 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:59:18.812848 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:59:18.812848 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:59:18.812848 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:59:18.812848 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:59:18.812848 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 18 04:59:18.812848 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 18 04:59:18.812848 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 18 04:59:18.813927 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 18 04:59:18.813927 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:59:18.813927 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:59:18.813927 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:59:18.813927 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:59:18.813927 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:59:18.813927 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:59:18.813927 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:59:18.813927 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jun 18 04:59:18.813927 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:59:18.813927 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:59:18.813927 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:59:18.813927 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:59:18.813927 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:59:18.813927 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:59:18.813927 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:59:18.813927 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:59:18.815066 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 18 04:59:18.815066 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 18 04:59:18.815066 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 18 04:59:18.815066 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 18 04:59:18.815066 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 18 04:59:18.815066 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 18 04:59:18.815066 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 18 04:59:18.815066 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 04:59:18.816802 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 382/754] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Jun 18 04:59:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:59:18.822676 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84467496-fb90-4f4d-86c4-783fceaf6e4d None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:18.847621 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:18.896013 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f99928-07a3-4c8f-a81b-0a66e74c0930 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:18.935701 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1594f800-953d-46c8-9606-21666c58e23c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:18.936646 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1594f800-953d-46c8-9606-21666c58e23c None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:18.937022 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1594f800-953d-46c8-9606-21666c58e23c None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:18.937434 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1594f800-953d-46c8-9606-21666c58e23c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:18.948933 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1594f800-953d-46c8-9606-21666c58e23c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:18.950573 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1594f800-953d-46c8-9606-21666c58e23c None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:18.963487 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84467496-fb90-4f4d-86c4-783fceaf6e4d None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:18.969775 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application [None req-1594f800-953d-46c8-9606-21666c58e23c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 04:59:18.969775 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:18.969775 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:59:18.969775 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:59:18.969775 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:59:18.969775 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:59:18.969775 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:59:18.969775 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:59:18.969775 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:59:18.969775 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:59:18.969775 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:59:18.969775 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:59:18.969775 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 18 04:59:18.969775 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 18 04:59:18.969775 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 18 04:59:18.970826 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 18 04:59:18.970826 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:59:18.970826 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:59:18.970826 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:59:18.970826 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:59:18.970826 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:59:18.970826 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:59:18.970826 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:59:18.970826 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application with Lock( Jun 18 04:59:18.970826 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:59:18.970826 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:59:18.970826 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:59:18.970826 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:59:18.970826 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:59:18.970826 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:59:18.970826 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:59:18.970826 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:59:18.971917 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 18 04:59:18.971917 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 18 04:59:18.971917 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 04:59:18.971917 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 04:59:18.973596 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 373/755] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Jun 18 04:59:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:59:18.975005 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1594f800-953d-46c8-9606-21666c58e23c None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:19.010787 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d27f1ae-519a-470d-b7ae-37104da914f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:19.012050 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d27f1ae-519a-470d-b7ae-37104da914f0 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:19.013259 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d27f1ae-519a-470d-b7ae-37104da914f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:19.013765 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d27f1ae-519a-470d-b7ae-37104da914f0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:19.028496 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d27f1ae-519a-470d-b7ae-37104da914f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:19.030555 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d27f1ae-519a-470d-b7ae-37104da914f0 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:19.042172 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:19.053720 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-5d27f1ae-519a-470d-b7ae-37104da914f0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 04:59:19.053720 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:19.053720 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:59:19.053720 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:59:19.053720 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:59:19.053720 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:59:19.053720 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:59:19.053720 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:59:19.053720 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:59:19.053720 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:59:19.053720 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:59:19.053720 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:59:19.053720 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 18 04:59:19.053720 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 18 04:59:19.053720 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 18 04:59:19.064100 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 18 04:59:19.064100 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:59:19.064100 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:59:19.064100 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:59:19.064100 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:59:19.064100 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:59:19.064100 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:59:19.064100 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:59:19.064100 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jun 18 04:59:19.064100 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:59:19.064100 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:59:19.064100 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:59:19.064100 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:59:19.064100 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:59:19.064100 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:59:19.064100 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:59:19.064100 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:59:19.065197 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 18 04:59:19.065197 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 18 04:59:19.065197 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 04:59:19.065197 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 04:59:19.065197 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 383/756] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Jun 18 04:59:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:59:19.065197 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d27f1ae-519a-470d-b7ae-37104da914f0 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:19.097676 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1594f800-953d-46c8-9606-21666c58e23c None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:19.139244 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:19.143427 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d3532b-d533-4445-bdbd-cf22b8e2eb79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:19.144298 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d3532b-d533-4445-bdbd-cf22b8e2eb79 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:19.144545 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d3532b-d533-4445-bdbd-cf22b8e2eb79 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:19.144920 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d3532b-d533-4445-bdbd-cf22b8e2eb79 None admin] PATH_INFO: `/v3/projects` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:19.146225 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d3532b-d533-4445-bdbd-cf22b8e2eb79 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:19.146472 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d3532b-d533-4445-bdbd-cf22b8e2eb79 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:19.147964 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d3532b-d533-4445-bdbd-cf22b8e2eb79 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:19.165476 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 374/757] 10.222.0.23 () {56 vars in 1148 bytes} [Sat Jun 18 04:59:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:19.166492 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d3532b-d533-4445-bdbd-cf22b8e2eb79 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:19.181048 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d27f1ae-519a-470d-b7ae-37104da914f0 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:19.223821 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bf001d-b173-477b-ac2b-44ef29824d5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:19.225396 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bf001d-b173-477b-ac2b-44ef29824d5a None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:19.225961 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bf001d-b173-477b-ac2b-44ef29824d5a None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:19.226338 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bf001d-b173-477b-ac2b-44ef29824d5a None admin] PATH_INFO: `/v3/domains` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:19.227371 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bf001d-b173-477b-ac2b-44ef29824d5a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:19.227739 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bf001d-b173-477b-ac2b-44ef29824d5a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:19.229300 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bf001d-b173-477b-ac2b-44ef29824d5a None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:19.243331 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 384/758] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Jun 18 04:59:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:19.248524 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bf001d-b173-477b-ac2b-44ef29824d5a None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:19.305039 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d3532b-d533-4445-bdbd-cf22b8e2eb79 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:19.335484 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:19.351189 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596602bb-748c-4a19-af12-fa743bc459ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:19.352185 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596602bb-748c-4a19-af12-fa743bc459ca None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:19.352863 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596602bb-748c-4a19-af12-fa743bc459ca None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:19.353282 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596602bb-748c-4a19-af12-fa743bc459ca None admin] PATH_INFO: `/v3/domains` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:19.354843 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596602bb-748c-4a19-af12-fa743bc459ca None admin] RBAC: Adding query filter params (name=Default) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:19.355341 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596602bb-748c-4a19-af12-fa743bc459ca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:19.357893 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596602bb-748c-4a19-af12-fa743bc459ca None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:19.359990 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bf001d-b173-477b-ac2b-44ef29824d5a None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:19.377404 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 375/759] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Jun 18 04:59:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:19.378624 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596602bb-748c-4a19-af12-fa743bc459ca None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:19.401855 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:19.410470 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae37eba-95a2-4cc8-9e7c-2b450ebb44c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:19.411124 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae37eba-95a2-4cc8-9e7c-2b450ebb44c3 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:19.411412 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae37eba-95a2-4cc8-9e7c-2b450ebb44c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:19.412241 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae37eba-95a2-4cc8-9e7c-2b450ebb44c3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:19.413031 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae37eba-95a2-4cc8-9e7c-2b450ebb44c3 None admin] RBAC: Adding query filter params (role.id=21c65f2c51bd4dd283d13507d2714467, scope.project.id=c95f8f3084c5472cb8364b7c57207394, user.id=16b49e866398400580ae48aa5a96fc04) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:19.413347 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae37eba-95a2-4cc8-9e7c-2b450ebb44c3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:19.414822 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae37eba-95a2-4cc8-9e7c-2b450ebb44c3 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:19.420432 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 385/760] 10.222.0.23 () {56 vars in 1374 bytes} [Sat Jun 18 04:59:19 2022] GET /identity/v3/role_assignments?user.id=16b49e866398400580ae48aa5a96fc04&scope.project.id=c95f8f3084c5472cb8364b7c57207394&role.id=21c65f2c51bd4dd283d13507d2714467 => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:19.433464 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae37eba-95a2-4cc8-9e7c-2b450ebb44c3 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:19.502410 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596602bb-748c-4a19-af12-fa743bc459ca None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:19.541104 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4897fa-646d-4221-b25c-16c729e8268f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:19.542274 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4897fa-646d-4221-b25c-16c729e8268f None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:19.542702 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4897fa-646d-4221-b25c-16c729e8268f None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:19.543115 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4897fa-646d-4221-b25c-16c729e8268f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:19.551578 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4897fa-646d-4221-b25c-16c729e8268f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:19.553971 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4897fa-646d-4221-b25c-16c729e8268f None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:19.555029 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae37eba-95a2-4cc8-9e7c-2b450ebb44c3 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:19.570230 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application [None req-ec4897fa-646d-4221-b25c-16c729e8268f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 18 04:59:19.570230 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:19.570230 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:59:19.570230 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:59:19.570230 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:59:19.570230 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:59:19.570230 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:59:19.570230 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:59:19.570230 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:59:19.570230 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:59:19.570230 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:59:19.570230 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:59:19.570230 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 18 04:59:19.570230 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 18 04:59:19.570230 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 18 04:59:19.571318 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 18 04:59:19.571318 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:59:19.571318 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:59:19.571318 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:59:19.571318 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:59:19.571318 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:59:19.571318 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:59:19.571318 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:59:19.571318 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application with Lock( Jun 18 04:59:19.571318 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:59:19.571318 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:59:19.571318 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:59:19.571318 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:59:19.571318 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:59:19.571318 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:59:19.571318 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:59:19.571318 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:59:19.572338 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 18 04:59:19.572338 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 18 04:59:19.572338 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 18 04:59:19.572338 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 18 04:59:19.572338 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 18 04:59:19.572338 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 18 04:59:19.572338 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 18 04:59:19.572338 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 04:59:19.574690 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 376/761] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Jun 18 04:59:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:59:19.576158 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c911a7f1-debd-4b3f-9ef1-052776f90084 None None] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:19.576698 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c911a7f1-debd-4b3f-9ef1-052776f90084 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:19.577114 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c911a7f1-debd-4b3f-9ef1-052776f90084 None None] PATH_INFO: `/` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:19.578915 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 377/762] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 04:59:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 04:59:19.579854 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c911a7f1-debd-4b3f-9ef1-052776f90084 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:19.595160 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8000ed52-5df8-4ff4-8c00-261a2db30fe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:19.595950 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8000ed52-5df8-4ff4-8c00-261a2db30fe0 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:19.596308 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8000ed52-5df8-4ff4-8c00-261a2db30fe0 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:19.596558 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8000ed52-5df8-4ff4-8c00-261a2db30fe0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:19.606250 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8000ed52-5df8-4ff4-8c00-261a2db30fe0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:19.607722 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8000ed52-5df8-4ff4-8c00-261a2db30fe0 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:19.626295 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-8000ed52-5df8-4ff4-8c00-261a2db30fe0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 18 04:59:19.626295 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:19.626295 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:59:19.626295 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:59:19.626295 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:59:19.626295 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:59:19.626295 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:59:19.626295 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:59:19.626295 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:59:19.626295 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:59:19.626295 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:59:19.626295 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:59:19.626295 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 18 04:59:19.626295 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 18 04:59:19.626295 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 18 04:59:19.627372 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 18 04:59:19.627372 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:59:19.627372 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:59:19.627372 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:59:19.627372 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:59:19.627372 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:59:19.627372 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:59:19.627372 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:59:19.627372 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jun 18 04:59:19.627372 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:59:19.627372 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:59:19.627372 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:59:19.627372 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:59:19.627372 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:59:19.627372 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:59:19.627372 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:59:19.627372 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:59:19.628409 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 18 04:59:19.628409 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 18 04:59:19.628409 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 18 04:59:19.628409 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 18 04:59:19.628409 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 18 04:59:19.628409 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 18 04:59:19.628409 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 18 04:59:19.628409 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 04:59:19.628409 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 386/763] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Jun 18 04:59:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:59:19.628409 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1969e0ac-555f-45bf-9907-bb377b66e9fa None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:19.631742 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1969e0ac-555f-45bf-9907-bb377b66e9fa None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:19.631742 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1969e0ac-555f-45bf-9907-bb377b66e9fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:19.640563 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:19.651548 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-1969e0ac-555f-45bf-9907-bb377b66e9fa None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 04:59:19.663718 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-1969e0ac-555f-45bf-9907-bb377b66e9fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 04:59:19.704172 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c911a7f1-debd-4b3f-9ef1-052776f90084 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:19.705049 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:19.726609 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 387/764] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 04:59:19 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 18 04:59:19.727530 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1969e0ac-555f-45bf-9907-bb377b66e9fa None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:19.743852 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3362aee9-2e3f-4a26-a572-bd9041ba756f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:19.745003 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3362aee9-2e3f-4a26-a572-bd9041ba756f None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:19.745548 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3362aee9-2e3f-4a26-a572-bd9041ba756f None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:19.746029 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3362aee9-2e3f-4a26-a572-bd9041ba756f None admin] PATH_INFO: `/v3/projects` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:19.747049 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3362aee9-2e3f-4a26-a572-bd9041ba756f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:19.747580 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3362aee9-2e3f-4a26-a572-bd9041ba756f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:19.749229 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3362aee9-2e3f-4a26-a572-bd9041ba756f None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:19.764299 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 378/765] 10.222.0.23 () {56 vars in 1148 bytes} [Sat Jun 18 04:59:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:19.765502 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3362aee9-2e3f-4a26-a572-bd9041ba756f None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:19.875393 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1969e0ac-555f-45bf-9907-bb377b66e9fa None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:19.880056 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3362aee9-2e3f-4a26-a572-bd9041ba756f None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:19.883035 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:19.914919 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719bcf98-584f-4971-8935-d000e95fd390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:19.915407 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719bcf98-584f-4971-8935-d000e95fd390 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:19.915749 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719bcf98-584f-4971-8935-d000e95fd390 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:19.916072 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719bcf98-584f-4971-8935-d000e95fd390 None admin] PATH_INFO: `/v3/projects` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:19.916831 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719bcf98-584f-4971-8935-d000e95fd390 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:19.917149 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719bcf98-584f-4971-8935-d000e95fd390 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:19.920309 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719bcf98-584f-4971-8935-d000e95fd390 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:19.920768 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87eebf9d-add9-4629-99c3-b70fb71af8a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:19.921668 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eebf9d-add9-4629-99c3-b70fb71af8a9 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:19.921953 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eebf9d-add9-4629-99c3-b70fb71af8a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:19.922229 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eebf9d-add9-4629-99c3-b70fb71af8a9 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:19.922934 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87eebf9d-add9-4629-99c3-b70fb71af8a9 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:19.924564 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87eebf9d-add9-4629-99c3-b70fb71af8a9 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:19.935992 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 388/766] 10.222.0.23 () {56 vars in 1148 bytes} [Sat Jun 18 04:59:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:19.936827 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719bcf98-584f-4971-8935-d000e95fd390 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:19.940400 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application [None req-87eebf9d-add9-4629-99c3-b70fb71af8a9 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 18 04:59:19.940400 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:19.940400 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 18 04:59:19.940400 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 18 04:59:19.940400 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 18 04:59:19.940400 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 18 04:59:19.940400 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 18 04:59:19.940400 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 18 04:59:19.940400 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 18 04:59:19.940400 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 04:59:19.940400 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 18 04:59:19.940400 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 04:59:19.940400 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:19.940400 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:59:19.940400 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:59:19.940400 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:59:19.941326 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:59:19.941326 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:59:19.941326 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:59:19.941326 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:59:19.941326 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:59:19.941326 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:59:19.941326 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:59:19.941326 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 18 04:59:19.941326 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 18 04:59:19.941326 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 18 04:59:19.941326 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 18 04:59:19.941326 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:59:19.941326 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:59:19.941326 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:59:19.941326 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:59:19.941326 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:59:19.942194 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:59:19.942194 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:59:19.942194 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application with Lock( Jun 18 04:59:19.942194 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:59:19.942194 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:59:19.942194 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:59:19.942194 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:59:19.942194 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:59:19.942194 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:59:19.942194 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:59:19.942194 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:59:19.942194 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 18 04:59:19.942194 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 18 04:59:19.942194 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 18 04:59:19.942194 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 04:59:19.942194 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 379/767] 10.222.0.23 () {56 vars in 1129 bytes} [Sat Jun 18 04:59:19 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:59:19.944738 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87eebf9d-add9-4629-99c3-b70fb71af8a9 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:19.991957 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:20.066341 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719bcf98-584f-4971-8935-d000e95fd390 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:20.067815 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87eebf9d-add9-4629-99c3-b70fb71af8a9 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:20.105595 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4029d214-24ca-4186-8e5b-224b96f5874b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:20.106249 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4029d214-24ca-4186-8e5b-224b96f5874b None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:20.106532 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4029d214-24ca-4186-8e5b-224b96f5874b None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:20.106833 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4029d214-24ca-4186-8e5b-224b96f5874b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:20.107753 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4029d214-24ca-4186-8e5b-224b96f5874b None admin] RBAC: Adding query filter params (role.id=5ec7b68b793d4a54abd30460961346b9, scope.project.id=c95f8f3084c5472cb8364b7c57207394, user.id=e817fafd0c1b4bf9944481cf116f1260) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:20.108074 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4029d214-24ca-4186-8e5b-224b96f5874b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:20.109549 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4029d214-24ca-4186-8e5b-224b96f5874b None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:20.113703 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28e1b64-617d-485a-b145-15af0d7aa71d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:20.115540 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28e1b64-617d-485a-b145-15af0d7aa71d None admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:20.116282 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28e1b64-617d-485a-b145-15af0d7aa71d None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:20.116765 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28e1b64-617d-485a-b145-15af0d7aa71d None admin] PATH_INFO: `/v3/projects/c95f8f3084c5472cb8364b7c57207394/users/15e7f45ee0744c1d9602187eac8e8b80/roles/21c65f2c51bd4dd283d13507d2714467` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:20.116920 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 380/768] 10.222.0.23 () {56 vars in 1374 bytes} [Sat Jun 18 04:59:19 2022] GET /identity/v3/role_assignments?user.id=e817fafd0c1b4bf9944481cf116f1260&scope.project.id=c95f8f3084c5472cb8364b7c57207394&role.id=5ec7b68b793d4a54abd30460961346b9 => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:20.117870 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4029d214-24ca-4186-8e5b-224b96f5874b None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:20.119528 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28e1b64-617d-485a-b145-15af0d7aa71d None admin] RBAC: Authorizing `identity:create_grant(project_id=c95f8f3084c5472cb8364b7c57207394, user_id=15e7f45ee0744c1d9602187eac8e8b80, role_id=21c65f2c51bd4dd283d13507d2714467)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:20.122639 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28e1b64-617d-485a-b145-15af0d7aa71d None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:20.157643 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:20.239054 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4029d214-24ca-4186-8e5b-224b96f5874b None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:20.241857 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 389/769] 10.222.0.23 () {58 vars in 1439 bytes} [Sat Jun 18 04:59:19 2022] PUT /identity/v3/projects/c95f8f3084c5472cb8364b7c57207394/users/15e7f45ee0744c1d9602187eac8e8b80/roles/21c65f2c51bd4dd283d13507d2714467 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 04:59:20.274356 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbdcda4-dd76-40b4-a8c3-707a418592e4 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:20.274697 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbdcda4-dd76-40b4-a8c3-707a418592e4 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:20.274925 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbdcda4-dd76-40b4-a8c3-707a418592e4 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:20.276895 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 390/770] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 04:59:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 18 04:59:20.282802 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e16be15-9220-4021-a6f6-e0a21f0eb0aa None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:20.283121 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e16be15-9220-4021-a6f6-e0a21f0eb0aa None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:20.283369 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e16be15-9220-4021-a6f6-e0a21f0eb0aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:20.290513 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825b3477-ea0b-49ee-95d1-31d09721fb97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:20.291346 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825b3477-ea0b-49ee-95d1-31d09721fb97 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:20.291633 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825b3477-ea0b-49ee-95d1-31d09721fb97 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:20.291912 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825b3477-ea0b-49ee-95d1-31d09721fb97 None admin] PATH_INFO: `/v3/services` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:20.292645 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825b3477-ea0b-49ee-95d1-31d09721fb97 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:20.292956 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825b3477-ea0b-49ee-95d1-31d09721fb97 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:20.294446 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825b3477-ea0b-49ee-95d1-31d09721fb97 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:20.304416 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-0e16be15-9220-4021-a6f6-e0a21f0eb0aa None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 04:59:20.305984 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 381/771] 10.222.0.23 () {56 vars in 1124 bytes} [Sat Jun 18 04:59:20 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:20.306356 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-0e16be15-9220-4021-a6f6-e0a21f0eb0aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 04:59:20.314635 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825b3477-ea0b-49ee-95d1-31d09721fb97 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:20.449809 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 391/772] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 04:59:20 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 18 04:59:20.474996 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:20.476779 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e16be15-9220-4021-a6f6-e0a21f0eb0aa None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:20.487843 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825b3477-ea0b-49ee-95d1-31d09721fb97 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:20.515336 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:20.528643 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b8514c-c37f-4ad7-b244-2967dc1f055c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:20.529658 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b8514c-c37f-4ad7-b244-2967dc1f055c None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:20.530112 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b8514c-c37f-4ad7-b244-2967dc1f055c None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:20.530554 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b8514c-c37f-4ad7-b244-2967dc1f055c None admin] PATH_INFO: `/v3/services` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:20.531511 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b8514c-c37f-4ad7-b244-2967dc1f055c None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:20.533213 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b8514c-c37f-4ad7-b244-2967dc1f055c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:20.533504 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b8514c-c37f-4ad7-b244-2967dc1f055c None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:20.540517 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 382/773] 10.222.0.23 () {56 vars in 1124 bytes} [Sat Jun 18 04:59:20 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:20.546192 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b8514c-c37f-4ad7-b244-2967dc1f055c None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:20.658454 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e16be15-9220-4021-a6f6-e0a21f0eb0aa None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:20.700038 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b8514c-c37f-4ad7-b244-2967dc1f055c None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:20.723844 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5c6023-c171-4058-ada1-ae3a85feae4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:20.725126 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5c6023-c171-4058-ada1-ae3a85feae4c None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:20.725715 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5c6023-c171-4058-ada1-ae3a85feae4c None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:20.729922 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5c6023-c171-4058-ada1-ae3a85feae4c None admin] PATH_INFO: `/v3/services/network` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:20.729922 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5c6023-c171-4058-ada1-ae3a85feae4c None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:20.729922 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5c6023-c171-4058-ada1-ae3a85feae4c None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:20.757205 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-5b5c6023-c171-4058-ada1-ae3a85feae4c None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 18 04:59:20.757205 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:20.757205 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 18 04:59:20.757205 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 18 04:59:20.757205 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 18 04:59:20.757205 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 18 04:59:20.757205 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 18 04:59:20.757205 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 18 04:59:20.757205 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 18 04:59:20.757205 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 04:59:20.757205 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 18 04:59:20.757205 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 04:59:20.757205 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:20.757205 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:59:20.757205 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:59:20.757205 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:59:20.757205 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:59:20.758290 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:59:20.758290 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:59:20.758290 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:59:20.758290 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:59:20.758290 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:59:20.758290 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:59:20.758290 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 18 04:59:20.758290 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 18 04:59:20.758290 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 18 04:59:20.758290 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 18 04:59:20.758290 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:59:20.758290 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:59:20.758290 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:59:20.758290 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:59:20.758290 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:59:20.758290 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:59:20.759747 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:59:20.759747 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jun 18 04:59:20.759747 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:59:20.759747 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:59:20.759747 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:59:20.759747 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:59:20.759747 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:59:20.759747 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:59:20.759747 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:59:20.759747 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:59:20.759747 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 18 04:59:20.759747 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 18 04:59:20.759747 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 18 04:59:20.759747 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 04:59:20.759747 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 392/774] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Jun 18 04:59:20 2022] GET /identity/v3/services/network => generated 88 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:59:20.764360 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b136ab-f86f-48fb-bea7-709271018f9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:20.765030 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b136ab-f86f-48fb-bea7-709271018f9a None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:20.765317 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b136ab-f86f-48fb-bea7-709271018f9a None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:20.765676 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b136ab-f86f-48fb-bea7-709271018f9a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:20.766419 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b136ab-f86f-48fb-bea7-709271018f9a None admin] RBAC: Adding query filter params (interface=public, service_id=14119fd3cc594dbcb1da85228c5f4aa9, region_id=RegionOne) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:20.766734 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b136ab-f86f-48fb-bea7-709271018f9a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:20.768141 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b136ab-f86f-48fb-bea7-709271018f9a None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:20.770340 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5c6023-c171-4058-ada1-ae3a85feae4c None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:20.788113 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 383/775] 10.222.0.23 () {56 vars in 1251 bytes} [Sat Jun 18 04:59:20 2022] GET /identity/v3/endpoints?service_id=14119fd3cc594dbcb1da85228c5f4aa9&interface=public®ion_id=RegionOne => generated 194 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 18 04:59:20.793849 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b136ab-f86f-48fb-bea7-709271018f9a None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:20.853017 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:20.958930 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:20.961624 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b136ab-f86f-48fb-bea7-709271018f9a None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:20.979647 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5c6023-c171-4058-ada1-ae3a85feae4c None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:21.024993 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9caeb5ed-e1ab-4639-b154-e9a6023c20fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:21.025611 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caeb5ed-e1ab-4639-b154-e9a6023c20fe None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:21.025899 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caeb5ed-e1ab-4639-b154-e9a6023c20fe None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:21.026180 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caeb5ed-e1ab-4639-b154-e9a6023c20fe None admin] PATH_INFO: `/v3/services` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:21.026873 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9caeb5ed-e1ab-4639-b154-e9a6023c20fe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:21.028247 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9caeb5ed-e1ab-4639-b154-e9a6023c20fe None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:21.043401 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fb96b6-7286-445e-8115-cf1bec41e09d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:21.044062 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fb96b6-7286-445e-8115-cf1bec41e09d None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:21.044345 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fb96b6-7286-445e-8115-cf1bec41e09d None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:21.044621 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fb96b6-7286-445e-8115-cf1bec41e09d None admin] PATH_INFO: `/v3/services` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:21.045327 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fb96b6-7286-445e-8115-cf1bec41e09d None admin] RBAC: Adding query filter params (name=network) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:21.052186 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 384/776] 10.222.0.23 () {56 vars in 1087 bytes} [Sat Jun 18 04:59:20 2022] GET /identity/v3/services => generated 772 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 18 04:59:21.053680 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fb96b6-7286-445e-8115-cf1bec41e09d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:21.054789 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fb96b6-7286-445e-8115-cf1bec41e09d None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:21.060399 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 393/777] 10.222.0.23 () {56 vars in 1112 bytes} [Sat Jun 18 04:59:20 2022] GET /identity/v3/services?name=network => generated 124 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:21.066038 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9caeb5ed-e1ab-4639-b154-e9a6023c20fe None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:21.196732 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:21.219680 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9caeb5ed-e1ab-4639-b154-e9a6023c20fe None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:21.266505 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dfbb5e-e8b9-4767-ad6a-e1cbc50d6369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:21.267182 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dfbb5e-e8b9-4767-ad6a-e1cbc50d6369 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:21.267510 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dfbb5e-e8b9-4767-ad6a-e1cbc50d6369 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:21.267790 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dfbb5e-e8b9-4767-ad6a-e1cbc50d6369 None admin] PATH_INFO: `/v3/services` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:21.268566 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36dfbb5e-e8b9-4767-ad6a-e1cbc50d6369 None admin] RBAC: Adding query filter params (type=network) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:21.268882 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36dfbb5e-e8b9-4767-ad6a-e1cbc50d6369 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:21.270287 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36dfbb5e-e8b9-4767-ad6a-e1cbc50d6369 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:21.282357 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 385/778] 10.222.0.23 () {56 vars in 1112 bytes} [Sat Jun 18 04:59:21 2022] GET /identity/v3/services?type=network => generated 350 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:21.287936 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fb96b6-7286-445e-8115-cf1bec41e09d None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:21.417635 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fb96b6-7286-445e-8115-cf1bec41e09d None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:21.453848 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd98871-67da-4ded-9091-afc6dc8d5abc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:21.454525 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd98871-67da-4ded-9091-afc6dc8d5abc None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:21.454813 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd98871-67da-4ded-9091-afc6dc8d5abc None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:21.455087 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd98871-67da-4ded-9091-afc6dc8d5abc None admin] PATH_INFO: `/v3/endpoints` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:21.455885 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd98871-67da-4ded-9091-afc6dc8d5abc None admin] RBAC: Adding query filter params (interface=public, service_id=7d16875a0e25449d81cf37b3f13f5476, region_id=RegionOne) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:21.456200 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd98871-67da-4ded-9091-afc6dc8d5abc None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:21.457602 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd98871-67da-4ded-9091-afc6dc8d5abc None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:21.466296 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 394/779] 10.222.0.23 () {56 vars in 1251 bytes} [Sat Jun 18 04:59:21 2022] GET /identity/v3/endpoints?service_id=7d16875a0e25449d81cf37b3f13f5476&interface=public®ion_id=RegionOne => generated 194 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:21.471808 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dfbb5e-e8b9-4767-ad6a-e1cbc50d6369 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:21.578312 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dfbb5e-e8b9-4767-ad6a-e1cbc50d6369 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:21.614458 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ea5e30-2264-4272-94ea-cd945eeea5f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:21.615980 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ea5e30-2264-4272-94ea-cd945eeea5f9 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:21.616270 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ea5e30-2264-4272-94ea-cd945eeea5f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:21.616544 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ea5e30-2264-4272-94ea-cd945eeea5f9 None admin] PATH_INFO: `/v3/services` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:21.617233 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ea5e30-2264-4272-94ea-cd945eeea5f9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:21.618634 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ea5e30-2264-4272-94ea-cd945eeea5f9 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:21.624533 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 386/780] 10.222.0.23 () {56 vars in 1087 bytes} [Sat Jun 18 04:59:21 2022] GET /identity/v3/services => generated 772 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:21.780411 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb097843-0c66-4f7a-9686-53eb94d2ad41 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:21.780739 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb097843-0c66-4f7a-9686-53eb94d2ad41 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:21.780980 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb097843-0c66-4f7a-9686-53eb94d2ad41 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:21.782958 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 395/781] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 04:59:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 04:59:21.788627 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b06488-56d7-4688-894b-97df535408f0 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:21.789477 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b06488-56d7-4688-894b-97df535408f0 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:21.789933 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b06488-56d7-4688-894b-97df535408f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:21.807614 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-84b06488-56d7-4688-894b-97df535408f0 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 04:59:21.809057 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-84b06488-56d7-4688-894b-97df535408f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 04:59:21.819868 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:21.885569 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 387/782] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 04:59:21 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 18 04:59:21.903474 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb097843-0c66-4f7a-9686-53eb94d2ad41 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:21.932735 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:22.026661 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb097843-0c66-4f7a-9686-53eb94d2ad41 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:22.069739 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41be3f90-f5ac-4854-a5db-80cf7ab2944f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:22.070541 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41be3f90-f5ac-4854-a5db-80cf7ab2944f None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:22.070824 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41be3f90-f5ac-4854-a5db-80cf7ab2944f None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:22.071097 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41be3f90-f5ac-4854-a5db-80cf7ab2944f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:22.090334 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41be3f90-f5ac-4854-a5db-80cf7ab2944f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 18 04:59:22.098112 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-41be3f90-f5ac-4854-a5db-80cf7ab2944f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 18 04:59:22.098112 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:22.098112 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:59:22.098112 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:59:22.098112 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:59:22.098112 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:59:22.098112 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:59:22.098112 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:59:22.098112 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:59:22.098112 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:59:22.098112 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:59:22.098112 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:59:22.098112 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 18 04:59:22.098112 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 18 04:59:22.098112 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 18 04:59:22.098112 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 18 04:59:22.099061 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 18 04:59:22.099061 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 18 04:59:22.099061 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 18 04:59:22.099061 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 18 04:59:22.099061 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:59:22.099061 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:59:22.099061 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:59:22.099061 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:59:22.099061 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:59:22.099061 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:59:22.099061 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:59:22.099061 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jun 18 04:59:22.099061 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:59:22.099061 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:59:22.099061 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:59:22.099061 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:59:22.100550 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:59:22.100550 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:59:22.100550 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:59:22.100550 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:59:22.100550 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 18 04:59:22.100550 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 18 04:59:22.100550 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 18 04:59:22.100550 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 18 04:59:22.100550 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 18 04:59:22.100550 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 18 04:59:22.100550 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 18 04:59:22.100550 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 04:59:22.100550 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 396/783] 10.222.0.23 () {56 vars in 1096 bytes} [Sat Jun 18 04:59:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:59:22.106499 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b06488-56d7-4688-894b-97df535408f0 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:22.269528 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b06488-56d7-4688-894b-97df535408f0 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:22.322093 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b774e3-2a7e-4636-8cff-c3a1c8bb91b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:22.322766 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b774e3-2a7e-4636-8cff-c3a1c8bb91b1 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:22.323045 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b774e3-2a7e-4636-8cff-c3a1c8bb91b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:22.323317 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b774e3-2a7e-4636-8cff-c3a1c8bb91b1 None admin] PATH_INFO: `/v3/roles` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:22.324066 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b774e3-2a7e-4636-8cff-c3a1c8bb91b1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:22.324371 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b774e3-2a7e-4636-8cff-c3a1c8bb91b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:22.325748 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b774e3-2a7e-4636-8cff-c3a1c8bb91b1 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:22.335530 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 388/784] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Jun 18 04:59:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 18 04:59:22.341085 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41be3f90-f5ac-4854-a5db-80cf7ab2944f None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:22.394068 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00beac9-562b-4549-b1b7-8818108e2236 None None] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:22.394352 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00beac9-562b-4549-b1b7-8818108e2236 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:22.394628 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00beac9-562b-4549-b1b7-8818108e2236 None None] PATH_INFO: `/` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:22.397720 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 389/785] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 04:59:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 04:59:22.402068 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78a4f07-de7e-414d-86c7-4de634bf1cdc None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:22.402467 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78a4f07-de7e-414d-86c7-4de634bf1cdc None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:22.402755 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78a4f07-de7e-414d-86c7-4de634bf1cdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:22.422042 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:22.436493 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-a78a4f07-de7e-414d-86c7-4de634bf1cdc None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 04:59:22.437824 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-a78a4f07-de7e-414d-86c7-4de634bf1cdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 04:59:22.446063 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:22.462179 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41be3f90-f5ac-4854-a5db-80cf7ab2944f None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:22.498693 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e4c0ca-b8fe-4054-90df-9111c5c8fb3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:22.499492 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e4c0ca-b8fe-4054-90df-9111c5c8fb3c None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:22.499778 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e4c0ca-b8fe-4054-90df-9111c5c8fb3c None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:22.500059 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e4c0ca-b8fe-4054-90df-9111c5c8fb3c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:22.512763 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e4c0ca-b8fe-4054-90df-9111c5c8fb3c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:22.514327 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e4c0ca-b8fe-4054-90df-9111c5c8fb3c None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:22.533144 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-a3e4c0ca-b8fe-4054-90df-9111c5c8fb3c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 04:59:22.533144 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:22.533144 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:59:22.533144 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:59:22.533144 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:59:22.533144 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:59:22.533144 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:59:22.533144 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:59:22.533144 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:59:22.533144 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:59:22.533144 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:59:22.533144 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:59:22.533144 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 18 04:59:22.533144 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 18 04:59:22.533144 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 18 04:59:22.534936 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 18 04:59:22.534936 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:59:22.534936 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:59:22.534936 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:59:22.534936 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:59:22.534936 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:59:22.534936 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:59:22.534936 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:59:22.534936 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jun 18 04:59:22.534936 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:59:22.534936 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:59:22.534936 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:59:22.534936 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:59:22.534936 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:59:22.534936 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:59:22.534936 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:59:22.534936 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:59:22.536573 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 18 04:59:22.536573 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 18 04:59:22.536573 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 04:59:22.536573 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 04:59:22.536573 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 397/786] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Jun 18 04:59:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:59:22.546516 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e4c0ca-b8fe-4054-90df-9111c5c8fb3c None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:22.558566 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 390/787] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 04:59:22 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 18 04:59:22.590012 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78a4f07-de7e-414d-86c7-4de634bf1cdc None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:22.672893 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e4c0ca-b8fe-4054-90df-9111c5c8fb3c None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:22.712508 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7eb4506-43d1-425c-8806-d4d837ccc4ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:22.715586 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eb4506-43d1-425c-8806-d4d837ccc4ac None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:22.716192 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eb4506-43d1-425c-8806-d4d837ccc4ac None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:22.716486 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eb4506-43d1-425c-8806-d4d837ccc4ac None admin] PATH_INFO: `/v3/domains` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:22.717224 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7eb4506-43d1-425c-8806-d4d837ccc4ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:22.717580 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7eb4506-43d1-425c-8806-d4d837ccc4ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:22.718961 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7eb4506-43d1-425c-8806-d4d837ccc4ac None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:22.724976 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:22.740952 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 398/788] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Jun 18 04:59:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:22.746479 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7eb4506-43d1-425c-8806-d4d837ccc4ac None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:22.800824 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78a4f07-de7e-414d-86c7-4de634bf1cdc None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:22.837849 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a1a4d3-2854-4a26-8451-f975a423dc97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:22.838758 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a1a4d3-2854-4a26-8451-f975a423dc97 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:22.839137 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a1a4d3-2854-4a26-8451-f975a423dc97 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:22.839482 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a1a4d3-2854-4a26-8451-f975a423dc97 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:22.850534 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a1a4d3-2854-4a26-8451-f975a423dc97 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 18 04:59:22.859473 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application [None req-f2a1a4d3-2854-4a26-8451-f975a423dc97 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 18 04:59:22.859473 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:22.859473 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:59:22.859473 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:59:22.859473 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:59:22.859473 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:59:22.859473 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:59:22.859473 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:59:22.859473 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:59:22.859473 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:59:22.859473 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:59:22.859473 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:59:22.859473 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 18 04:59:22.859473 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 18 04:59:22.859473 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 18 04:59:22.859473 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 18 04:59:22.860398 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 18 04:59:22.860398 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 18 04:59:22.860398 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 18 04:59:22.860398 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 18 04:59:22.860398 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:59:22.860398 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:59:22.860398 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:59:22.860398 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:59:22.860398 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:59:22.860398 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:59:22.860398 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:59:22.860398 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application with Lock( Jun 18 04:59:22.860398 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:59:22.860398 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:59:22.860398 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:59:22.860398 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:59:22.861224 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:59:22.861224 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:59:22.861224 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:59:22.861224 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:59:22.861224 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 18 04:59:22.861224 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 18 04:59:22.861224 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 18 04:59:22.861224 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 18 04:59:22.861224 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 18 04:59:22.861224 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 18 04:59:22.861224 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 18 04:59:22.861224 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 04:59:22.862756 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:22.866309 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 391/789] 10.222.0.23 () {56 vars in 1102 bytes} [Sat Jun 18 04:59:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 18 04:59:22.867633 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab09f84-20fa-4fc1-b1d6-b7f0c15e4909 None None] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:22.868026 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab09f84-20fa-4fc1-b1d6-b7f0c15e4909 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:22.868358 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab09f84-20fa-4fc1-b1d6-b7f0c15e4909 None None] PATH_INFO: `/` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:22.871194 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 392/790] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 04:59:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 04:59:22.871721 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7eb4506-43d1-425c-8806-d4d837ccc4ac None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:22.871914 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab09f84-20fa-4fc1-b1d6-b7f0c15e4909 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:22.918490 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd231609-e4d5-4201-a5f8-a91143a914b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:22.920113 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd231609-e4d5-4201-a5f8-a91143a914b7 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:22.920397 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd231609-e4d5-4201-a5f8-a91143a914b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:22.920674 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd231609-e4d5-4201-a5f8-a91143a914b7 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:22.930099 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd231609-e4d5-4201-a5f8-a91143a914b7 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:22.932101 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd231609-e4d5-4201-a5f8-a91143a914b7 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:22.950183 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-dd231609-e4d5-4201-a5f8-a91143a914b7 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 18 04:59:22.950183 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:22.950183 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:59:22.950183 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:59:22.950183 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:59:22.950183 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:59:22.950183 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:59:22.950183 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:59:22.950183 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:59:22.950183 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:59:22.950183 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:59:22.950183 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:59:22.950183 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 18 04:59:22.950183 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 18 04:59:22.950183 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 18 04:59:22.950183 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 18 04:59:22.951200 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:59:22.951200 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:59:22.951200 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 18 04:59:22.951200 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 18 04:59:22.951200 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 18 04:59:22.951200 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 18 04:59:22.951200 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:59:22.951200 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:59:22.951200 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:59:22.951200 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:59:22.951200 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:59:22.951200 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jun 18 04:59:22.951200 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:59:22.951200 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:59:22.951200 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:59:22.951200 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:59:22.951200 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:59:22.951200 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:59:22.952924 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:59:22.952924 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:59:22.952924 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 18 04:59:22.952924 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 18 04:59:22.952924 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 18 04:59:22.952924 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 18 04:59:22.952924 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 18 04:59:22.952924 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 18 04:59:22.952924 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 18 04:59:22.952924 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 04:59:22.952924 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 399/791] 10.222.0.23 () {56 vars in 1096 bytes} [Sat Jun 18 04:59:22 2022] GET /identity/v3/users/swift => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 18 04:59:22.953929 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e1563d-335f-4aa2-88a1-ed3eb1d5d7fb None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:22.953929 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e1563d-335f-4aa2-88a1-ed3eb1d5d7fb None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:22.954175 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e1563d-335f-4aa2-88a1-ed3eb1d5d7fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:22.982468 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-42e1563d-335f-4aa2-88a1-ed3eb1d5d7fb None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 04:59:22.984382 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-42e1563d-335f-4aa2-88a1-ed3eb1d5d7fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 04:59:22.991399 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab09f84-20fa-4fc1-b1d6-b7f0c15e4909 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:23.030092 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961c84fa-ce3b-402c-87e0-2fe6d7c951dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:23.030920 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961c84fa-ce3b-402c-87e0-2fe6d7c951dd None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:23.031213 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961c84fa-ce3b-402c-87e0-2fe6d7c951dd None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:23.031582 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961c84fa-ce3b-402c-87e0-2fe6d7c951dd None admin] PATH_INFO: `/v3/roles` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:23.032347 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961c84fa-ce3b-402c-87e0-2fe6d7c951dd None admin] RBAC: Adding query filter params (name=service) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:23.032659 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961c84fa-ce3b-402c-87e0-2fe6d7c951dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:23.034142 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961c84fa-ce3b-402c-87e0-2fe6d7c951dd None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:23.047243 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 393/792] 10.222.0.23 () {56 vars in 1103 bytes} [Sat Jun 18 04:59:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:23.048118 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961c84fa-ce3b-402c-87e0-2fe6d7c951dd None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:23.060701 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:23.081611 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 400/793] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 04:59:22 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 18 04:59:23.082786 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e1563d-335f-4aa2-88a1-ed3eb1d5d7fb None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:23.128190 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:23.175532 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961c84fa-ce3b-402c-87e0-2fe6d7c951dd None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:23.217128 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2a4156-405f-4a50-81f2-6439fbb9581c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:23.218133 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2a4156-405f-4a50-81f2-6439fbb9581c None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:23.218564 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2a4156-405f-4a50-81f2-6439fbb9581c None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:23.219012 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2a4156-405f-4a50-81f2-6439fbb9581c None admin] PATH_INFO: `/v3/users` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:23.219989 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2a4156-405f-4a50-81f2-6439fbb9581c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:23.220450 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2a4156-405f-4a50-81f2-6439fbb9581c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:23.222144 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2a4156-405f-4a50-81f2-6439fbb9581c None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:23.231682 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e1563d-335f-4aa2-88a1-ed3eb1d5d7fb None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:23.242245 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 394/794] 10.222.0.23 () {56 vars in 1135 bytes} [Sat Jun 18 04:59:23 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:23.243432 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2a4156-405f-4a50-81f2-6439fbb9581c None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:23.277823 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df22987d-e180-4617-8671-fef676e3e98b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:23.278621 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df22987d-e180-4617-8671-fef676e3e98b None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:23.278899 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df22987d-e180-4617-8671-fef676e3e98b None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:23.279170 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df22987d-e180-4617-8671-fef676e3e98b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:23.292124 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df22987d-e180-4617-8671-fef676e3e98b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:23.293690 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df22987d-e180-4617-8671-fef676e3e98b None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:23.315275 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-df22987d-e180-4617-8671-fef676e3e98b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 04:59:23.315275 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:23.315275 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:59:23.315275 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:59:23.315275 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:59:23.315275 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:59:23.315275 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:59:23.315275 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:59:23.315275 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:59:23.315275 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:59:23.315275 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:59:23.315275 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:59:23.315275 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 18 04:59:23.315275 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 18 04:59:23.315275 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 18 04:59:23.316158 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 18 04:59:23.316158 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:59:23.316158 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:59:23.316158 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:59:23.316158 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:59:23.316158 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:59:23.316158 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:59:23.316158 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:59:23.316158 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jun 18 04:59:23.316158 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:59:23.316158 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:59:23.316158 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:59:23.316158 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:59:23.316158 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:59:23.316158 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:59:23.316158 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:59:23.316158 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:59:23.317030 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 18 04:59:23.317030 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 18 04:59:23.317030 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 04:59:23.317030 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 04:59:23.323855 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 401/795] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Jun 18 04:59:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:59:23.324911 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df22987d-e180-4617-8671-fef676e3e98b None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:23.365535 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2a4156-405f-4a50-81f2-6439fbb9581c None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:23.367434 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:23.386147 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:23.409793 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af5287d-cd24-4d48-aa60-cd645a93cf82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:23.412322 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af5287d-cd24-4d48-aa60-cd645a93cf82 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:23.412797 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af5287d-cd24-4d48-aa60-cd645a93cf82 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:23.413248 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af5287d-cd24-4d48-aa60-cd645a93cf82 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:23.429578 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af5287d-cd24-4d48-aa60-cd645a93cf82 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 18 04:59:23.437209 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application [None req-6af5287d-cd24-4d48-aa60-cd645a93cf82 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 18 04:59:23.437209 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:23.437209 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:59:23.437209 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:59:23.437209 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:59:23.437209 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:59:23.437209 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:59:23.437209 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:59:23.437209 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:59:23.437209 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:59:23.437209 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:59:23.437209 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:59:23.437209 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 18 04:59:23.437209 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 18 04:59:23.437209 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 18 04:59:23.437209 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 18 04:59:23.438269 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 18 04:59:23.438269 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 18 04:59:23.438269 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 18 04:59:23.438269 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 18 04:59:23.438269 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:59:23.438269 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:59:23.438269 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:59:23.438269 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:59:23.438269 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:59:23.438269 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:59:23.438269 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:59:23.438269 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application with Lock( Jun 18 04:59:23.438269 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:59:23.438269 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:59:23.438269 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:59:23.438269 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:59:23.439108 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:59:23.439108 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:59:23.439108 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:59:23.439108 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:59:23.439108 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 18 04:59:23.439108 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 18 04:59:23.439108 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 18 04:59:23.439108 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 18 04:59:23.439108 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 18 04:59:23.439108 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 18 04:59:23.439108 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 18 04:59:23.439108 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 04:59:23.440558 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 395/796] 10.222.0.23 () {56 vars in 1096 bytes} [Sat Jun 18 04:59:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:59:23.441020 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af5287d-cd24-4d48-aa60-cd645a93cf82 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:23.461712 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df22987d-e180-4617-8671-fef676e3e98b None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:23.500260 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901eba86-a4a3-4e89-aee8-9eec4429ee46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:23.501107 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901eba86-a4a3-4e89-aee8-9eec4429ee46 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:23.501417 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901eba86-a4a3-4e89-aee8-9eec4429ee46 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:23.501711 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901eba86-a4a3-4e89-aee8-9eec4429ee46 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:23.512633 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901eba86-a4a3-4e89-aee8-9eec4429ee46 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:23.514149 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901eba86-a4a3-4e89-aee8-9eec4429ee46 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:23.533425 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-901eba86-a4a3-4e89-aee8-9eec4429ee46 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 04:59:23.533425 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:23.533425 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:59:23.533425 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:59:23.533425 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:59:23.533425 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:59:23.533425 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:59:23.533425 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:59:23.533425 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:59:23.533425 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:59:23.533425 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:59:23.533425 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:59:23.533425 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 18 04:59:23.533425 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 18 04:59:23.533425 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 18 04:59:23.534356 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 18 04:59:23.534356 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:59:23.534356 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:59:23.534356 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:59:23.534356 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:59:23.534356 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:59:23.534356 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:59:23.534356 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:59:23.534356 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jun 18 04:59:23.534356 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:59:23.534356 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:59:23.534356 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:59:23.534356 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:59:23.534356 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:59:23.534356 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:59:23.534356 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:59:23.534356 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:59:23.535736 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 18 04:59:23.535736 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 18 04:59:23.535736 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 04:59:23.535736 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 04:59:23.535736 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 402/797] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Jun 18 04:59:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:59:23.536019 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901eba86-a4a3-4e89-aee8-9eec4429ee46 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:23.564425 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af5287d-cd24-4d48-aa60-cd645a93cf82 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:23.602065 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a97677-c6bd-4c97-88c7-67091d599ac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:23.602753 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a97677-c6bd-4c97-88c7-67091d599ac8 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:23.603044 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a97677-c6bd-4c97-88c7-67091d599ac8 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:23.603323 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a97677-c6bd-4c97-88c7-67091d599ac8 None admin] PATH_INFO: `/v3/domains` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:23.604057 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a97677-c6bd-4c97-88c7-67091d599ac8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:23.604369 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a97677-c6bd-4c97-88c7-67091d599ac8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:23.605860 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a97677-c6bd-4c97-88c7-67091d599ac8 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:23.619321 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 396/798] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Jun 18 04:59:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:23.620486 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a97677-c6bd-4c97-88c7-67091d599ac8 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:23.637732 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:23.647577 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901eba86-a4a3-4e89-aee8-9eec4429ee46 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:23.664238 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:23.684268 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8a9458-c72a-4253-a17d-2234df7b03b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:23.684941 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8a9458-c72a-4253-a17d-2234df7b03b0 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:23.685232 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8a9458-c72a-4253-a17d-2234df7b03b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:23.685544 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8a9458-c72a-4253-a17d-2234df7b03b0 None admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:23.686297 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8a9458-c72a-4253-a17d-2234df7b03b0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:23.686614 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8a9458-c72a-4253-a17d-2234df7b03b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:23.688131 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8a9458-c72a-4253-a17d-2234df7b03b0 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:23.701417 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 403/799] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Jun 18 04:59:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:23.701417 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8a9458-c72a-4253-a17d-2234df7b03b0 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:23.737967 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a97677-c6bd-4c97-88c7-67091d599ac8 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:23.776496 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940b0628-5c23-4205-bb87-e275149ac854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:23.777572 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940b0628-5c23-4205-bb87-e275149ac854 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:23.778061 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940b0628-5c23-4205-bb87-e275149ac854 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:23.778554 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940b0628-5c23-4205-bb87-e275149ac854 None admin] PATH_INFO: `/v3/domains` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:23.779475 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940b0628-5c23-4205-bb87-e275149ac854 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:23.779994 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940b0628-5c23-4205-bb87-e275149ac854 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:23.781788 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940b0628-5c23-4205-bb87-e275149ac854 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:23.794494 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 397/800] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Jun 18 04:59:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:23.800099 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940b0628-5c23-4205-bb87-e275149ac854 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:23.807666 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8a9458-c72a-4253-a17d-2234df7b03b0 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:23.845154 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4369509d-b063-41f7-8b79-31ed99feaf8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:23.846000 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4369509d-b063-41f7-8b79-31ed99feaf8e None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:23.846285 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4369509d-b063-41f7-8b79-31ed99feaf8e None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:23.846562 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4369509d-b063-41f7-8b79-31ed99feaf8e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:23.856221 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4369509d-b063-41f7-8b79-31ed99feaf8e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:23.857679 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4369509d-b063-41f7-8b79-31ed99feaf8e None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:23.873781 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-4369509d-b063-41f7-8b79-31ed99feaf8e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 18 04:59:23.873781 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:23.873781 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:59:23.873781 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:59:23.873781 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:59:23.873781 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:59:23.873781 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:59:23.873781 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:59:23.873781 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:59:23.873781 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:59:23.873781 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:59:23.873781 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:59:23.873781 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 18 04:59:23.873781 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 18 04:59:23.873781 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 18 04:59:23.873781 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 18 04:59:23.874764 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:59:23.874764 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:59:23.874764 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 18 04:59:23.874764 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 18 04:59:23.874764 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 18 04:59:23.874764 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 18 04:59:23.874764 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:59:23.874764 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:59:23.874764 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:59:23.874764 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:59:23.874764 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:59:23.874764 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jun 18 04:59:23.874764 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:59:23.874764 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:59:23.874764 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:59:23.874764 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:59:23.874764 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:59:23.874764 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:59:23.875702 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:59:23.875702 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:59:23.875702 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 18 04:59:23.875702 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 18 04:59:23.875702 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 18 04:59:23.875702 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 18 04:59:23.875702 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 18 04:59:23.875702 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 18 04:59:23.875702 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 18 04:59:23.875702 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 04:59:23.879715 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 404/801] 10.222.0.23 () {56 vars in 1117 bytes} [Sat Jun 18 04:59:23 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:59:23.881149 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4369509d-b063-41f7-8b79-31ed99feaf8e None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:23.896954 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:23.916548 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940b0628-5c23-4205-bb87-e275149ac854 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:23.928472 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:23.955196 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08492d4f-2a2a-4707-a2d3-c8e63cfe54da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:23.956491 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08492d4f-2a2a-4707-a2d3-c8e63cfe54da None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:23.956997 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08492d4f-2a2a-4707-a2d3-c8e63cfe54da None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:23.957524 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08492d4f-2a2a-4707-a2d3-c8e63cfe54da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:23.972304 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08492d4f-2a2a-4707-a2d3-c8e63cfe54da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:23.973909 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08492d4f-2a2a-4707-a2d3-c8e63cfe54da None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:23.991741 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4369509d-b063-41f7-8b79-31ed99feaf8e None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:23.993563 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application [None req-08492d4f-2a2a-4707-a2d3-c8e63cfe54da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 04:59:23.993563 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:23.993563 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:59:23.993563 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:59:23.993563 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:59:23.993563 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:59:23.993563 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:59:23.993563 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:59:23.993563 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:59:23.993563 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:59:23.993563 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:59:23.993563 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:59:23.993563 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 18 04:59:23.993563 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 18 04:59:23.993563 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 18 04:59:23.994439 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 18 04:59:23.994439 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:59:23.994439 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:59:23.994439 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:59:23.994439 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:59:23.994439 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:59:23.994439 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:59:23.994439 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:59:23.994439 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application with Lock( Jun 18 04:59:23.994439 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:59:23.994439 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:59:23.994439 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:59:23.994439 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:59:23.994439 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:59:23.994439 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:59:23.994439 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:59:23.994439 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:59:23.995333 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 18 04:59:23.995333 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 18 04:59:23.995333 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 04:59:23.995333 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 04:59:23.995333 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 398/802] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Jun 18 04:59:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:59:23.996433 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08492d4f-2a2a-4707-a2d3-c8e63cfe54da None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:24.028062 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96125c87-1c1c-4a41-adb9-9271df2bd25d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:24.028767 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96125c87-1c1c-4a41-adb9-9271df2bd25d None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:24.029052 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96125c87-1c1c-4a41-adb9-9271df2bd25d None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:24.029333 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96125c87-1c1c-4a41-adb9-9271df2bd25d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:24.036276 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96125c87-1c1c-4a41-adb9-9271df2bd25d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:24.037795 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96125c87-1c1c-4a41-adb9-9271df2bd25d None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:24.055467 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-96125c87-1c1c-4a41-adb9-9271df2bd25d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 18 04:59:24.055467 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:24.055467 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:59:24.055467 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:59:24.055467 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:59:24.055467 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:59:24.055467 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:59:24.055467 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:59:24.055467 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:59:24.055467 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:59:24.055467 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:59:24.055467 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:59:24.055467 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 18 04:59:24.055467 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 18 04:59:24.055467 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 18 04:59:24.056409 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 18 04:59:24.056409 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:59:24.056409 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:59:24.056409 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:59:24.056409 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:59:24.056409 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:59:24.056409 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:59:24.056409 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:59:24.056409 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jun 18 04:59:24.056409 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:59:24.056409 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:59:24.056409 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:59:24.056409 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:59:24.056409 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:59:24.056409 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:59:24.056409 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:59:24.056409 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:59:24.057321 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 18 04:59:24.057321 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 18 04:59:24.057321 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 18 04:59:24.057321 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 18 04:59:24.057321 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 18 04:59:24.057321 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 18 04:59:24.057321 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 18 04:59:24.057321 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 04:59:24.057321 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 405/803] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Jun 18 04:59:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:59:24.058563 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96125c87-1c1c-4a41-adb9-9271df2bd25d None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:24.116029 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08492d4f-2a2a-4707-a2d3-c8e63cfe54da None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:24.134726 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:24.152648 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00850336-88f9-4789-968c-a36ce156bc5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:24.153313 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00850336-88f9-4789-968c-a36ce156bc5b None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:24.153631 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00850336-88f9-4789-968c-a36ce156bc5b None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:24.153912 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00850336-88f9-4789-968c-a36ce156bc5b None admin] PATH_INFO: `/v3/users` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:24.154745 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00850336-88f9-4789-968c-a36ce156bc5b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:24.155061 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00850336-88f9-4789-968c-a36ce156bc5b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:24.156489 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00850336-88f9-4789-968c-a36ce156bc5b None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:24.164556 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96125c87-1c1c-4a41-adb9-9271df2bd25d None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:24.173951 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 399/804] 10.222.0.23 () {56 vars in 1149 bytes} [Sat Jun 18 04:59:23 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:24.175484 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3bb638-c5b6-44ad-a36d-bf040dbe370f None None] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:24.175781 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3bb638-c5b6-44ad-a36d-bf040dbe370f None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:24.176059 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3bb638-c5b6-44ad-a36d-bf040dbe370f None None] PATH_INFO: `/` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:24.177802 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 400/805] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 04:59:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 04:59:24.178498 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3bb638-c5b6-44ad-a36d-bf040dbe370f None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:24.186872 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:24.202350 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61a70ff-7831-4608-8b12-ee4cf5ccae9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:24.203008 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61a70ff-7831-4608-8b12-ee4cf5ccae9d None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:24.203312 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61a70ff-7831-4608-8b12-ee4cf5ccae9d None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:24.203597 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61a70ff-7831-4608-8b12-ee4cf5ccae9d None admin] PATH_INFO: `/v3/domains` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:24.204312 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61a70ff-7831-4608-8b12-ee4cf5ccae9d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:24.204681 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61a70ff-7831-4608-8b12-ee4cf5ccae9d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:24.206094 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61a70ff-7831-4608-8b12-ee4cf5ccae9d None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:24.217663 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 406/806] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Jun 18 04:59:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:24.218897 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58daf5e5-ed13-4d2c-8d9a-3e65185e4f95 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:24.219182 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58daf5e5-ed13-4d2c-8d9a-3e65185e4f95 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:24.219468 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58daf5e5-ed13-4d2c-8d9a-3e65185e4f95 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:24.220681 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 407/807] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 04:59:24 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 04:59:24.221364 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58daf5e5-ed13-4d2c-8d9a-3e65185e4f95 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:24.283233 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3bb638-c5b6-44ad-a36d-bf040dbe370f None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:24.316699 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e70b7df-f28d-4ed6-92cc-fe86b3af0156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:24.317337 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e70b7df-f28d-4ed6-92cc-fe86b3af0156 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:24.317656 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e70b7df-f28d-4ed6-92cc-fe86b3af0156 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:24.318468 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e70b7df-f28d-4ed6-92cc-fe86b3af0156 None admin] PATH_INFO: `/v3/projects` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:24.319429 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e70b7df-f28d-4ed6-92cc-fe86b3af0156 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:24.319740 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e70b7df-f28d-4ed6-92cc-fe86b3af0156 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:24.321146 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e70b7df-f28d-4ed6-92cc-fe86b3af0156 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:24.323318 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58daf5e5-ed13-4d2c-8d9a-3e65185e4f95 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:24.332408 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 401/808] 10.222.0.23 () {56 vars in 1148 bytes} [Sat Jun 18 04:59:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:24.333761 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82284b55-ac48-48f3-a415-9b534fe4ea0c None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:24.334104 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82284b55-ac48-48f3-a415-9b534fe4ea0c None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:24.334376 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82284b55-ac48-48f3-a415-9b534fe4ea0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:24.350895 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-82284b55-ac48-48f3-a415-9b534fe4ea0c None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 04:59:24.352087 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-82284b55-ac48-48f3-a415-9b534fe4ea0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 04:59:24.352944 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:24.357863 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88845273-b6f8-4b38-a2f6-e213818e755d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:24.358638 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88845273-b6f8-4b38-a2f6-e213818e755d None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:24.358918 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88845273-b6f8-4b38-a2f6-e213818e755d None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:24.359201 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88845273-b6f8-4b38-a2f6-e213818e755d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:24.370812 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88845273-b6f8-4b38-a2f6-e213818e755d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:24.372221 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88845273-b6f8-4b38-a2f6-e213818e755d None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:24.389331 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-88845273-b6f8-4b38-a2f6-e213818e755d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 04:59:24.389331 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:24.389331 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:59:24.389331 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:59:24.389331 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:59:24.389331 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:59:24.389331 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:59:24.389331 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:59:24.389331 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:59:24.389331 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:59:24.389331 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:59:24.389331 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:59:24.389331 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 18 04:59:24.389331 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 18 04:59:24.389331 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 18 04:59:24.390269 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 18 04:59:24.390269 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:59:24.390269 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:59:24.390269 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:59:24.390269 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:59:24.390269 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:59:24.390269 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:59:24.390269 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:59:24.390269 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jun 18 04:59:24.390269 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:59:24.390269 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:59:24.390269 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:59:24.390269 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:59:24.390269 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:59:24.390269 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:59:24.390269 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:59:24.390269 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:59:24.391180 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 18 04:59:24.391180 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 18 04:59:24.391180 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 04:59:24.391180 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 04:59:24.391180 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 408/809] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Jun 18 04:59:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:59:24.391619 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88845273-b6f8-4b38-a2f6-e213818e755d None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:24.414273 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:24.426818 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 402/810] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 04:59:24 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 18 04:59:24.426818 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85effefc-8999-4d4a-97fa-b04bdef54c09 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:24.426818 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85effefc-8999-4d4a-97fa-b04bdef54c09 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:24.426818 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85effefc-8999-4d4a-97fa-b04bdef54c09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:24.445913 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-85effefc-8999-4d4a-97fa-b04bdef54c09 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 04:59:24.447334 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-85effefc-8999-4d4a-97fa-b04bdef54c09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 04:59:24.498182 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88845273-b6f8-4b38-a2f6-e213818e755d None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:24.514332 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 403/811] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 04:59:24 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 18 04:59:24.515144 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85effefc-8999-4d4a-97fa-b04bdef54c09 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:24.537265 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c983826d-f422-40bc-a0e6-b23f359667fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:24.538091 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c983826d-f422-40bc-a0e6-b23f359667fe None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:24.538374 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c983826d-f422-40bc-a0e6-b23f359667fe None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:24.538649 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c983826d-f422-40bc-a0e6-b23f359667fe None admin] PATH_INFO: `/v3/users/nova` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:24.546179 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c983826d-f422-40bc-a0e6-b23f359667fe None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:24.547637 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c983826d-f422-40bc-a0e6-b23f359667fe None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:24.561673 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-c983826d-f422-40bc-a0e6-b23f359667fe None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 18 04:59:24.561673 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:24.561673 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:59:24.561673 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:59:24.561673 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:59:24.561673 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:59:24.561673 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:59:24.561673 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:59:24.561673 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:59:24.561673 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:59:24.561673 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:59:24.561673 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:59:24.561673 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 18 04:59:24.561673 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 18 04:59:24.561673 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 18 04:59:24.561673 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 18 04:59:24.562770 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:59:24.562770 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:59:24.562770 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 18 04:59:24.562770 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 18 04:59:24.562770 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 18 04:59:24.562770 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 18 04:59:24.562770 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:59:24.562770 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:59:24.562770 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:59:24.562770 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:59:24.562770 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:59:24.562770 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jun 18 04:59:24.562770 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:59:24.562770 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:59:24.562770 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:59:24.562770 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:59:24.562770 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:59:24.562770 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:59:24.569056 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:59:24.569056 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:59:24.569056 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 18 04:59:24.569056 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 18 04:59:24.569056 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 18 04:59:24.569056 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 18 04:59:24.569056 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 18 04:59:24.569056 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 18 04:59:24.569056 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 18 04:59:24.569056 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 04:59:24.569056 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 409/812] 10.222.0.23 () {56 vars in 1093 bytes} [Sat Jun 18 04:59:24 2022] GET /identity/v3/users/nova => generated 82 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:59:24.569056 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c983826d-f422-40bc-a0e6-b23f359667fe None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:24.608860 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:24.621613 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85effefc-8999-4d4a-97fa-b04bdef54c09 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:24.647927 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:24.656874 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b53a5e-7283-4d7e-aefa-1e45c93f621f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:24.657798 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b53a5e-7283-4d7e-aefa-1e45c93f621f None admin] REQUEST_METHOD: `PUT` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:24.658081 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b53a5e-7283-4d7e-aefa-1e45c93f621f None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:24.658359 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b53a5e-7283-4d7e-aefa-1e45c93f621f None admin] PATH_INFO: `/v3/projects/c95f8f3084c5472cb8364b7c57207394/users/16b49e866398400580ae48aa5a96fc04/roles/21c65f2c51bd4dd283d13507d2714467` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:24.660324 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b53a5e-7283-4d7e-aefa-1e45c93f621f None admin] RBAC: Authorizing `identity:create_grant(project_id=c95f8f3084c5472cb8364b7c57207394, user_id=16b49e866398400580ae48aa5a96fc04, role_id=21c65f2c51bd4dd283d13507d2714467)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:24.662344 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b53a5e-7283-4d7e-aefa-1e45c93f621f None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:24.663457 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c983826d-f422-40bc-a0e6-b23f359667fe None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:24.696788 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc27ffd4-0ab6-4437-acd2-649a2b0446ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:24.697460 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc27ffd4-0ab6-4437-acd2-649a2b0446ca None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:24.697741 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc27ffd4-0ab6-4437-acd2-649a2b0446ca None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:24.698011 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc27ffd4-0ab6-4437-acd2-649a2b0446ca None admin] PATH_INFO: `/v3/domains` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:24.698716 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc27ffd4-0ab6-4437-acd2-649a2b0446ca None admin] RBAC: Adding query filter params (name=Default) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:24.699022 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc27ffd4-0ab6-4437-acd2-649a2b0446ca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:24.700362 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc27ffd4-0ab6-4437-acd2-649a2b0446ca None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:24.712384 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 410/813] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Jun 18 04:59:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:24.713245 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc27ffd4-0ab6-4437-acd2-649a2b0446ca None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:24.757035 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 404/814] 10.222.0.23 () {58 vars in 1439 bytes} [Sat Jun 18 04:59:24 2022] PUT /identity/v3/projects/c95f8f3084c5472cb8364b7c57207394/users/16b49e866398400580ae48aa5a96fc04/roles/21c65f2c51bd4dd283d13507d2714467 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 04:59:24.757891 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b53a5e-7283-4d7e-aefa-1e45c93f621f None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:24.823190 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc27ffd4-0ab6-4437-acd2-649a2b0446ca None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:24.825838 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:24.860518 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b7cf98-7df8-4f61-8dc3-605ff6c11d0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:24.861341 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b7cf98-7df8-4f61-8dc3-605ff6c11d0a None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:24.861658 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b7cf98-7df8-4f61-8dc3-605ff6c11d0a None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:24.861929 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b7cf98-7df8-4f61-8dc3-605ff6c11d0a None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:24.862599 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b7cf98-7df8-4f61-8dc3-605ff6c11d0a None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:24.863998 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b7cf98-7df8-4f61-8dc3-605ff6c11d0a None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:24.878817 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-14b7cf98-7df8-4f61-8dc3-605ff6c11d0a None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 18 04:59:24.878817 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:24.878817 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 18 04:59:24.878817 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 18 04:59:24.878817 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 18 04:59:24.878817 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 18 04:59:24.878817 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 18 04:59:24.878817 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 18 04:59:24.878817 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 18 04:59:24.878817 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 04:59:24.878817 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 18 04:59:24.878817 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 04:59:24.878817 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:24.878817 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:59:24.878817 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:59:24.878817 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:59:24.879997 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:59:24.879997 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:59:24.879997 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:59:24.879997 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:59:24.879997 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:59:24.879997 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:59:24.879997 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:59:24.879997 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 18 04:59:24.879997 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 18 04:59:24.879997 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 18 04:59:24.879997 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 18 04:59:24.879997 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:59:24.879997 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:59:24.879997 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:59:24.879997 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:59:24.879997 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:59:24.881012 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:59:24.881012 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:59:24.881012 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jun 18 04:59:24.881012 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:59:24.881012 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:59:24.881012 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:59:24.881012 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:59:24.881012 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:59:24.881012 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:59:24.881012 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:59:24.881012 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:59:24.881012 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 18 04:59:24.881012 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 18 04:59:24.881012 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 18 04:59:24.881012 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 04:59:24.881012 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 411/815] 10.222.0.23 () {56 vars in 1129 bytes} [Sat Jun 18 04:59:24 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:59:24.882039 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b7cf98-7df8-4f61-8dc3-605ff6c11d0a None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:24.894473 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b53a5e-7283-4d7e-aefa-1e45c93f621f None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:24.940438 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc2494b-0af4-4081-a35e-c433f0b0de56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:24.941293 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc2494b-0af4-4081-a35e-c433f0b0de56 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:24.941622 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc2494b-0af4-4081-a35e-c433f0b0de56 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:24.941901 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc2494b-0af4-4081-a35e-c433f0b0de56 None admin] PATH_INFO: `/v3/services/network` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:24.942563 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc2494b-0af4-4081-a35e-c433f0b0de56 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:24.945185 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc2494b-0af4-4081-a35e-c433f0b0de56 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:24.957254 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application [None req-0bc2494b-0af4-4081-a35e-c433f0b0de56 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 18 04:59:24.957254 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:24.957254 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 18 04:59:24.957254 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 18 04:59:24.957254 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 18 04:59:24.957254 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 18 04:59:24.957254 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 18 04:59:24.957254 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 18 04:59:24.957254 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 18 04:59:24.957254 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 04:59:24.957254 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 18 04:59:24.957254 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 04:59:24.957254 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:24.957254 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:59:24.957254 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:59:24.957254 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:59:24.957254 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:59:24.958275 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:59:24.958275 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:59:24.958275 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:59:24.958275 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:59:24.958275 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:59:24.958275 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:59:24.958275 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 18 04:59:24.958275 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 18 04:59:24.958275 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 18 04:59:24.958275 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 18 04:59:24.958275 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:59:24.958275 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:59:24.958275 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:59:24.958275 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:59:24.958275 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:59:24.958275 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:59:24.959793 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:59:24.959793 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application with Lock( Jun 18 04:59:24.959793 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:59:24.959793 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:59:24.959793 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:59:24.959793 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:59:24.959793 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:59:24.959793 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:59:24.959793 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:59:24.959793 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:59:24.959793 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 18 04:59:24.959793 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 18 04:59:24.959793 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 18 04:59:24.959793 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 04:59:24.959793 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 405/816] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Jun 18 04:59:24 2022] GET /identity/v3/services/network => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:59:24.960572 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc2494b-0af4-4081-a35e-c433f0b0de56 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:25.001294 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b7cf98-7df8-4f61-8dc3-605ff6c11d0a None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:25.013810 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:25.039460 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54afd711-5975-4ecf-9736-e804c4d2d18e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:25.040541 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54afd711-5975-4ecf-9736-e804c4d2d18e None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:25.040878 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54afd711-5975-4ecf-9736-e804c4d2d18e None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:25.041153 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54afd711-5975-4ecf-9736-e804c4d2d18e None admin] PATH_INFO: `/v3/users` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:25.042356 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54afd711-5975-4ecf-9736-e804c4d2d18e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:25.042679 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54afd711-5975-4ecf-9736-e804c4d2d18e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:25.044404 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54afd711-5975-4ecf-9736-e804c4d2d18e None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:25.066333 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc2494b-0af4-4081-a35e-c433f0b0de56 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:25.071307 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 412/817] 10.222.0.23 () {56 vars in 1133 bytes} [Sat Jun 18 04:59:24 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:25.071307 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54afd711-5975-4ecf-9736-e804c4d2d18e None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:25.099696 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:25.105525 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7578224-a98b-445e-9320-a91e901ce6ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:25.106317 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7578224-a98b-445e-9320-a91e901ce6ed None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:25.106603 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7578224-a98b-445e-9320-a91e901ce6ed None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:25.108713 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7578224-a98b-445e-9320-a91e901ce6ed None admin] PATH_INFO: `/v3/projects/service` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:25.114065 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7578224-a98b-445e-9320-a91e901ce6ed None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:25.116420 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7578224-a98b-445e-9320-a91e901ce6ed None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:25.131415 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application [None req-c7578224-a98b-445e-9320-a91e901ce6ed None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 18 04:59:25.131415 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:25.131415 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:59:25.131415 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:59:25.131415 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:59:25.131415 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:59:25.131415 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:59:25.131415 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:59:25.131415 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:59:25.131415 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:59:25.131415 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:59:25.131415 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:59:25.131415 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 18 04:59:25.131415 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 18 04:59:25.131415 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 18 04:59:25.132328 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 18 04:59:25.132328 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:59:25.132328 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:59:25.132328 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:59:25.132328 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:59:25.132328 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:59:25.132328 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:59:25.132328 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:59:25.132328 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application with Lock( Jun 18 04:59:25.132328 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:59:25.132328 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:59:25.132328 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:59:25.132328 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:59:25.132328 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:59:25.132328 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:59:25.132328 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:59:25.132328 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:59:25.133310 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 18 04:59:25.133310 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 18 04:59:25.133310 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 18 04:59:25.133310 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 18 04:59:25.133310 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 18 04:59:25.133310 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 18 04:59:25.133310 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 18 04:59:25.133310 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 04:59:25.133310 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 406/818] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Jun 18 04:59:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:59:25.133310 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7578224-a98b-445e-9320-a91e901ce6ed None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:25.184122 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54afd711-5975-4ecf-9736-e804c4d2d18e None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:25.223262 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49d66e1-22a5-4fec-b27b-ae61cb975a04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:25.224737 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49d66e1-22a5-4fec-b27b-ae61cb975a04 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:25.225480 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49d66e1-22a5-4fec-b27b-ae61cb975a04 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:25.226077 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49d66e1-22a5-4fec-b27b-ae61cb975a04 None admin] PATH_INFO: `/v3/services` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:25.226964 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49d66e1-22a5-4fec-b27b-ae61cb975a04 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:25.227429 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49d66e1-22a5-4fec-b27b-ae61cb975a04 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:25.229086 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49d66e1-22a5-4fec-b27b-ae61cb975a04 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:25.234785 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 413/819] 10.222.0.23 () {56 vars in 1124 bytes} [Sat Jun 18 04:59:25 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:25.235877 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49d66e1-22a5-4fec-b27b-ae61cb975a04 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:25.243053 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7578224-a98b-445e-9320-a91e901ce6ed None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:25.254064 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:25.280869 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4632cf27-b207-4afb-aaf4-d23af391015a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:25.281566 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4632cf27-b207-4afb-aaf4-d23af391015a None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:25.281849 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4632cf27-b207-4afb-aaf4-d23af391015a None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:25.282118 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4632cf27-b207-4afb-aaf4-d23af391015a None admin] PATH_INFO: `/v3/services` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:25.282816 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4632cf27-b207-4afb-aaf4-d23af391015a None admin] RBAC: Adding query filter params (name=network) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:25.283478 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4632cf27-b207-4afb-aaf4-d23af391015a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:25.284673 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4632cf27-b207-4afb-aaf4-d23af391015a None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:25.290916 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 407/820] 10.222.0.23 () {56 vars in 1112 bytes} [Sat Jun 18 04:59:25 2022] GET /identity/v3/services?name=network => generated 124 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:25.291780 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4632cf27-b207-4afb-aaf4-d23af391015a None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:25.340173 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:25.349830 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49d66e1-22a5-4fec-b27b-ae61cb975a04 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:25.391576 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e05923f-a6ad-4442-8247-f6eb2a8d60e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:25.392038 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e05923f-a6ad-4442-8247-f6eb2a8d60e4 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:25.392442 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e05923f-a6ad-4442-8247-f6eb2a8d60e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:25.392720 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e05923f-a6ad-4442-8247-f6eb2a8d60e4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:25.399670 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4632cf27-b207-4afb-aaf4-d23af391015a None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:25.438652 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2756c324-52cd-46bd-8083-c8849aeee313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:25.439470 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2756c324-52cd-46bd-8083-c8849aeee313 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:25.439754 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2756c324-52cd-46bd-8083-c8849aeee313 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:25.440032 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2756c324-52cd-46bd-8083-c8849aeee313 None admin] PATH_INFO: `/v3/projects` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:25.440818 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2756c324-52cd-46bd-8083-c8849aeee313 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:25.441133 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2756c324-52cd-46bd-8083-c8849aeee313 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:25.442607 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2756c324-52cd-46bd-8083-c8849aeee313 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:25.460517 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 408/821] 10.222.0.23 () {56 vars in 1148 bytes} [Sat Jun 18 04:59:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:25.460517 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2756c324-52cd-46bd-8083-c8849aeee313 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:25.470337 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e05923f-a6ad-4442-8247-f6eb2a8d60e4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:25.471883 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e05923f-a6ad-4442-8247-f6eb2a8d60e4 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:25.497617 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:25.565113 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:25.570634 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2756c324-52cd-46bd-8083-c8849aeee313 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:25.581178 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 414/822] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Jun 18 04:59:25 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 18 04:59:25.581178 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e05923f-a6ad-4442-8247-f6eb2a8d60e4 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:25.606229 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183599de-3ea2-40e8-bf7d-03fa95d9c766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:25.606882 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183599de-3ea2-40e8-bf7d-03fa95d9c766 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:25.607169 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183599de-3ea2-40e8-bf7d-03fa95d9c766 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:25.607444 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183599de-3ea2-40e8-bf7d-03fa95d9c766 None admin] PATH_INFO: `/v3/services` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:25.608144 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183599de-3ea2-40e8-bf7d-03fa95d9c766 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:25.608451 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183599de-3ea2-40e8-bf7d-03fa95d9c766 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:25.609851 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183599de-3ea2-40e8-bf7d-03fa95d9c766 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:25.614571 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 409/823] 10.222.0.23 () {56 vars in 1124 bytes} [Sat Jun 18 04:59:25 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:25.620943 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183599de-3ea2-40e8-bf7d-03fa95d9c766 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:25.686371 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e05923f-a6ad-4442-8247-f6eb2a8d60e4 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:25.726066 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:25.728075 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b0d1d6-269a-4f1c-924a-e48c11156cc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:25.728740 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b0d1d6-269a-4f1c-924a-e48c11156cc1 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:25.729024 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b0d1d6-269a-4f1c-924a-e48c11156cc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:25.729300 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b0d1d6-269a-4f1c-924a-e48c11156cc1 None admin] PATH_INFO: `/v3/services` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:25.730037 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b0d1d6-269a-4f1c-924a-e48c11156cc1 None admin] RBAC: Adding query filter params (type=network) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:25.730197 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183599de-3ea2-40e8-bf7d-03fa95d9c766 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:25.730369 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b0d1d6-269a-4f1c-924a-e48c11156cc1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:25.731712 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b0d1d6-269a-4f1c-924a-e48c11156cc1 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:25.737051 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 415/824] 10.222.0.23 () {56 vars in 1112 bytes} [Sat Jun 18 04:59:25 2022] GET /identity/v3/services?type=network => generated 350 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:25.740587 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b0d1d6-269a-4f1c-924a-e48c11156cc1 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:25.769141 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8033ab24-bc6e-413e-8f19-42d5afcaf4c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:25.770075 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8033ab24-bc6e-413e-8f19-42d5afcaf4c4 None admin] REQUEST_METHOD: `PUT` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:25.770359 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8033ab24-bc6e-413e-8f19-42d5afcaf4c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:25.770650 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8033ab24-bc6e-413e-8f19-42d5afcaf4c4 None admin] PATH_INFO: `/v3/projects/c95f8f3084c5472cb8364b7c57207394/users/e817fafd0c1b4bf9944481cf116f1260/roles/5ec7b68b793d4a54abd30460961346b9` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:25.786521 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8033ab24-bc6e-413e-8f19-42d5afcaf4c4 None admin] RBAC: Authorizing `identity:create_grant(project_id=c95f8f3084c5472cb8364b7c57207394, user_id=e817fafd0c1b4bf9944481cf116f1260, role_id=5ec7b68b793d4a54abd30460961346b9)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:25.788077 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8033ab24-bc6e-413e-8f19-42d5afcaf4c4 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:25.800635 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:25.844676 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b0d1d6-269a-4f1c-924a-e48c11156cc1 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:25.865508 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 410/825] 10.222.0.23 () {58 vars in 1439 bytes} [Sat Jun 18 04:59:25 2022] PUT /identity/v3/projects/c95f8f3084c5472cb8364b7c57207394/users/e817fafd0c1b4bf9944481cf116f1260/roles/5ec7b68b793d4a54abd30460961346b9 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 04:59:25.865508 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8033ab24-bc6e-413e-8f19-42d5afcaf4c4 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:25.895674 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622a74bd-0108-4e5f-8f1b-a5b0d5a0170d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:25.897256 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622a74bd-0108-4e5f-8f1b-a5b0d5a0170d None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:25.897605 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622a74bd-0108-4e5f-8f1b-a5b0d5a0170d None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:25.898174 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622a74bd-0108-4e5f-8f1b-a5b0d5a0170d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:25.916119 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622a74bd-0108-4e5f-8f1b-a5b0d5a0170d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:25.918283 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622a74bd-0108-4e5f-8f1b-a5b0d5a0170d None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:25.944453 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-622a74bd-0108-4e5f-8f1b-a5b0d5a0170d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 04:59:25.944453 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:25.944453 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:59:25.944453 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:59:25.944453 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:59:25.944453 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:59:25.944453 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:59:25.944453 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:59:25.944453 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:59:25.944453 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:59:25.944453 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:59:25.944453 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:59:25.944453 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 18 04:59:25.944453 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 18 04:59:25.944453 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 18 04:59:25.945551 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 18 04:59:25.945551 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:59:25.945551 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:59:25.945551 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:59:25.945551 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:59:25.945551 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:59:25.945551 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:59:25.945551 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:59:25.945551 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jun 18 04:59:25.945551 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:59:25.945551 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:59:25.945551 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:59:25.945551 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:59:25.945551 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:59:25.945551 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:59:25.945551 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:59:25.945551 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:59:25.946456 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 18 04:59:25.946456 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 18 04:59:25.946456 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 04:59:25.946456 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 04:59:25.949661 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 416/826] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Jun 18 04:59:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:59:25.950665 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622a74bd-0108-4e5f-8f1b-a5b0d5a0170d None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:26.001178 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8033ab24-bc6e-413e-8f19-42d5afcaf4c4 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:26.045487 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849a187a-e6c7-4c67-8294-b1d4914251a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:26.047183 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849a187a-e6c7-4c67-8294-b1d4914251a0 None admin] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:26.047183 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849a187a-e6c7-4c67-8294-b1d4914251a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:26.047183 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849a187a-e6c7-4c67-8294-b1d4914251a0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:26.047473 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849a187a-e6c7-4c67-8294-b1d4914251a0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:26.049857 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849a187a-e6c7-4c67-8294-b1d4914251a0 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:26.082759 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622a74bd-0108-4e5f-8f1b-a5b0d5a0170d None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:26.090493 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:26.117837 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6698b910-c55b-4d74-93a4-f6986db791f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:26.118492 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6698b910-c55b-4d74-93a4-f6986db791f7 None admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:26.118774 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6698b910-c55b-4d74-93a4-f6986db791f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:26.119047 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6698b910-c55b-4d74-93a4-f6986db791f7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:26.119790 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6698b910-c55b-4d74-93a4-f6986db791f7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:26.121165 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6698b910-c55b-4d74-93a4-f6986db791f7 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:26.152287 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 411/827] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Jun 18 04:59:25 2022] POST /identity/v3/endpoints => generated 346 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 04:59:26.153308 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849a187a-e6c7-4c67-8294-b1d4914251a0 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:26.162757 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:26.212914 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 417/828] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Jun 18 04:59:25 2022] POST /identity/v3/endpoints => generated 327 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 04:59:26.313744 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849a187a-e6c7-4c67-8294-b1d4914251a0 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:26.377334 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4bad86-4857-463a-a6ed-dd8e30904c55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:26.378454 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4bad86-4857-463a-a6ed-dd8e30904c55 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:26.378949 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4bad86-4857-463a-a6ed-dd8e30904c55 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:26.379440 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4bad86-4857-463a-a6ed-dd8e30904c55 None admin] PATH_INFO: `/v3/domains` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:26.380551 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4bad86-4857-463a-a6ed-dd8e30904c55 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:26.381312 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4bad86-4857-463a-a6ed-dd8e30904c55 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:26.383048 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4bad86-4857-463a-a6ed-dd8e30904c55 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:26.413331 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 412/829] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Jun 18 04:59:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:26.419078 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6698b910-c55b-4d74-93a4-f6986db791f7 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:26.537552 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6698b910-c55b-4d74-93a4-f6986db791f7 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:26.575939 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c037ee17-79dd-4749-9cea-4abc3e8e58d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:26.576744 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c037ee17-79dd-4749-9cea-4abc3e8e58d4 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:26.577026 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c037ee17-79dd-4749-9cea-4abc3e8e58d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:26.577300 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c037ee17-79dd-4749-9cea-4abc3e8e58d4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:26.585032 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c037ee17-79dd-4749-9cea-4abc3e8e58d4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:26.586563 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c037ee17-79dd-4749-9cea-4abc3e8e58d4 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:26.717350 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-c037ee17-79dd-4749-9cea-4abc3e8e58d4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 18 04:59:26.717350 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:26.717350 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:59:26.717350 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:59:26.717350 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:59:26.717350 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:59:26.717350 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:59:26.717350 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:59:26.717350 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:59:26.717350 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:59:26.717350 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:59:26.717350 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:59:26.717350 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 18 04:59:26.717350 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 18 04:59:26.717350 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 18 04:59:26.718348 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 18 04:59:26.718348 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:59:26.718348 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:59:26.718348 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:59:26.718348 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:59:26.718348 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:59:26.718348 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:59:26.718348 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:59:26.718348 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jun 18 04:59:26.718348 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:59:26.718348 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:59:26.718348 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:59:26.718348 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:59:26.718348 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:59:26.718348 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:59:26.718348 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:59:26.718348 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:59:26.719340 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 18 04:59:26.719340 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 18 04:59:26.719340 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 18 04:59:26.719340 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 18 04:59:26.719340 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 18 04:59:26.719340 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 18 04:59:26.719340 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 18 04:59:26.719340 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 04:59:26.719340 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 418/830] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Jun 18 04:59:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:59:26.725055 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4bad86-4857-463a-a6ed-dd8e30904c55 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:26.785919 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:26.833112 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4bad86-4857-463a-a6ed-dd8e30904c55 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:26.835397 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb94daaf-b608-46c0-a854-f906b641266c None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:26.835682 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb94daaf-b608-46c0-a854-f906b641266c None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:26.835958 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb94daaf-b608-46c0-a854-f906b641266c None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:26.837486 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 419/831] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 04:59:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 04:59:26.842677 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3a55a1-88db-44d3-b301-f0640ccf5786 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:26.843334 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3a55a1-88db-44d3-b301-f0640ccf5786 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:26.843880 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3a55a1-88db-44d3-b301-f0640ccf5786 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:26.862587 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-9b3a55a1-88db-44d3-b301-f0640ccf5786 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 04:59:26.864118 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-9b3a55a1-88db-44d3-b301-f0640ccf5786 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 04:59:26.882082 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15006543-ffdc-434b-91ca-0241ef8980e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:26.882748 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15006543-ffdc-434b-91ca-0241ef8980e5 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:26.883034 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15006543-ffdc-434b-91ca-0241ef8980e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:26.883312 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15006543-ffdc-434b-91ca-0241ef8980e5 None admin] PATH_INFO: `/v3/projects` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:26.884363 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:26.884580 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15006543-ffdc-434b-91ca-0241ef8980e5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:26.885613 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15006543-ffdc-434b-91ca-0241ef8980e5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:26.887495 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15006543-ffdc-434b-91ca-0241ef8980e5 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:26.904633 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 413/832] 10.222.0.23 () {56 vars in 1148 bytes} [Sat Jun 18 04:59:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:26.910985 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15006543-ffdc-434b-91ca-0241ef8980e5 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:26.947171 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 420/833] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 04:59:26 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 18 04:59:26.969599 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3a55a1-88db-44d3-b301-f0640ccf5786 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:27.041743 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15006543-ffdc-434b-91ca-0241ef8980e5 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:27.063857 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:27.076108 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3a55a1-88db-44d3-b301-f0640ccf5786 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:27.089801 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef2718c-d95a-4839-aacc-10dbf47672b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:27.090391 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef2718c-d95a-4839-aacc-10dbf47672b2 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:27.090744 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef2718c-d95a-4839-aacc-10dbf47672b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:27.091355 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef2718c-d95a-4839-aacc-10dbf47672b2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:27.091929 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef2718c-d95a-4839-aacc-10dbf47672b2 None admin] RBAC: Adding query filter params (role.id=21c65f2c51bd4dd283d13507d2714467, scope.project.id=c95f8f3084c5472cb8364b7c57207394, user.id=15e7f45ee0744c1d9602187eac8e8b80) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:27.092322 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef2718c-d95a-4839-aacc-10dbf47672b2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:27.094001 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef2718c-d95a-4839-aacc-10dbf47672b2 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:27.101647 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 414/834] 10.222.0.23 () {56 vars in 1374 bytes} [Sat Jun 18 04:59:26 2022] GET /identity/v3/role_assignments?user.id=15e7f45ee0744c1d9602187eac8e8b80&scope.project.id=c95f8f3084c5472cb8364b7c57207394&role.id=21c65f2c51bd4dd283d13507d2714467 => generated 608 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:27.122611 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695a3186-4c66-482b-a3e4-12cc1d0de2f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:27.123801 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695a3186-4c66-482b-a3e4-12cc1d0de2f2 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:27.124293 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695a3186-4c66-482b-a3e4-12cc1d0de2f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:27.124774 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695a3186-4c66-482b-a3e4-12cc1d0de2f2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:27.138476 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695a3186-4c66-482b-a3e4-12cc1d0de2f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 18 04:59:27.147688 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-695a3186-4c66-482b-a3e4-12cc1d0de2f2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 18 04:59:27.147688 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:27.147688 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:59:27.147688 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:59:27.147688 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:59:27.147688 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:59:27.147688 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:59:27.147688 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:59:27.147688 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:59:27.147688 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:59:27.147688 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:59:27.147688 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:59:27.147688 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 18 04:59:27.147688 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 18 04:59:27.147688 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 18 04:59:27.147688 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 18 04:59:27.148679 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 18 04:59:27.148679 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 18 04:59:27.148679 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 18 04:59:27.148679 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 18 04:59:27.148679 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:59:27.148679 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:59:27.148679 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:59:27.148679 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:59:27.148679 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:59:27.148679 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:59:27.148679 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:59:27.148679 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jun 18 04:59:27.148679 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:59:27.148679 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:59:27.148679 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:59:27.148679 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:59:27.149576 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:59:27.149576 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:59:27.149576 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:59:27.149576 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:59:27.149576 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 18 04:59:27.149576 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 18 04:59:27.149576 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 18 04:59:27.149576 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 18 04:59:27.149576 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 18 04:59:27.149576 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 18 04:59:27.149576 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 18 04:59:27.149576 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 04:59:27.152497 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 421/835] 10.222.0.23 () {56 vars in 1096 bytes} [Sat Jun 18 04:59:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:59:27.158988 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef2718c-d95a-4839-aacc-10dbf47672b2 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:27.275862 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef2718c-d95a-4839-aacc-10dbf47672b2 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:27.315985 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9891f7-8771-4cc2-be4f-ee9cfcc58e53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:27.316972 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9891f7-8771-4cc2-be4f-ee9cfcc58e53 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:27.317517 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9891f7-8771-4cc2-be4f-ee9cfcc58e53 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:27.318159 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9891f7-8771-4cc2-be4f-ee9cfcc58e53 None admin] PATH_INFO: `/v3/roles` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:27.319102 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9891f7-8771-4cc2-be4f-ee9cfcc58e53 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:27.319419 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9891f7-8771-4cc2-be4f-ee9cfcc58e53 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:27.320795 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9891f7-8771-4cc2-be4f-ee9cfcc58e53 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:27.329533 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 415/836] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Jun 18 04:59:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:27.335861 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695a3186-4c66-482b-a3e4-12cc1d0de2f2 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:27.352812 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:27.441176 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695a3186-4c66-482b-a3e4-12cc1d0de2f2 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:27.477598 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c84843-5eb5-4e42-8e91-5b74fddb03d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:27.478392 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c84843-5eb5-4e42-8e91-5b74fddb03d2 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:27.478677 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c84843-5eb5-4e42-8e91-5b74fddb03d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:27.478954 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c84843-5eb5-4e42-8e91-5b74fddb03d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:27.489369 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c84843-5eb5-4e42-8e91-5b74fddb03d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:27.491531 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c84843-5eb5-4e42-8e91-5b74fddb03d2 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:27.510137 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-f3c84843-5eb5-4e42-8e91-5b74fddb03d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 04:59:27.510137 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:27.510137 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:59:27.510137 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:59:27.510137 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:59:27.510137 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:59:27.510137 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:59:27.510137 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:59:27.510137 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:59:27.510137 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:59:27.510137 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:59:27.510137 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:59:27.510137 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 18 04:59:27.510137 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 18 04:59:27.510137 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 18 04:59:27.511061 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 18 04:59:27.511061 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:59:27.511061 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:59:27.511061 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:59:27.511061 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:59:27.511061 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:59:27.511061 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:59:27.511061 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:59:27.511061 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jun 18 04:59:27.511061 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:59:27.511061 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:59:27.511061 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:59:27.511061 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:59:27.511061 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:59:27.511061 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:59:27.511061 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:59:27.511061 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:59:27.511935 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 18 04:59:27.511935 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 18 04:59:27.511935 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 04:59:27.511935 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 04:59:27.511935 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 422/837] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Jun 18 04:59:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:59:27.518393 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9891f7-8771-4cc2-be4f-ee9cfcc58e53 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:27.682217 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:27.746634 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9891f7-8771-4cc2-be4f-ee9cfcc58e53 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:27.781734 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa18ace-3d7f-4315-a139-30f38c81e142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:27.782465 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa18ace-3d7f-4315-a139-30f38c81e142 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:27.782746 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa18ace-3d7f-4315-a139-30f38c81e142 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:27.783081 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa18ace-3d7f-4315-a139-30f38c81e142 None admin] PATH_INFO: `/v3/domains` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:27.783854 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa18ace-3d7f-4315-a139-30f38c81e142 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:27.784170 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa18ace-3d7f-4315-a139-30f38c81e142 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:27.785628 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa18ace-3d7f-4315-a139-30f38c81e142 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:27.798200 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 416/838] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Jun 18 04:59:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:27.806474 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c84843-5eb5-4e42-8e91-5b74fddb03d2 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:27.870295 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:27.908663 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c84843-5eb5-4e42-8e91-5b74fddb03d2 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:27.943009 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c861d3-3d4e-49f2-8405-f702994d29c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:27.943737 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c861d3-3d4e-49f2-8405-f702994d29c4 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:27.944019 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c861d3-3d4e-49f2-8405-f702994d29c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:27.944297 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c861d3-3d4e-49f2-8405-f702994d29c4 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:27.952241 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c861d3-3d4e-49f2-8405-f702994d29c4 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:27.953721 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c861d3-3d4e-49f2-8405-f702994d29c4 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:27.968338 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-d9c861d3-3d4e-49f2-8405-f702994d29c4 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 18 04:59:27.968338 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:27.968338 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:59:27.968338 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:59:27.968338 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:59:27.968338 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:59:27.968338 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:59:27.968338 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:59:27.968338 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:59:27.968338 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:59:27.968338 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:59:27.968338 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:59:27.968338 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 18 04:59:27.968338 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 18 04:59:27.968338 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 18 04:59:27.968338 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 18 04:59:27.969302 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:59:27.969302 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:59:27.969302 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 18 04:59:27.969302 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 18 04:59:27.969302 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 18 04:59:27.969302 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 18 04:59:27.969302 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:59:27.969302 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:59:27.969302 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:59:27.969302 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:59:27.969302 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:59:27.969302 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jun 18 04:59:27.969302 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:59:27.969302 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:59:27.969302 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:59:27.969302 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:59:27.969302 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:59:27.969302 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:59:27.970691 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:59:27.970691 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:59:27.970691 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 18 04:59:27.970691 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 18 04:59:27.970691 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 18 04:59:27.970691 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 18 04:59:27.970691 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 18 04:59:27.970691 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 18 04:59:27.970691 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 18 04:59:27.970691 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 04:59:27.970691 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 423/839] 10.222.0.23 () {56 vars in 1096 bytes} [Sat Jun 18 04:59:27 2022] GET /identity/v3/users/swift => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:59:27.975779 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa18ace-3d7f-4315-a139-30f38c81e142 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:28.069813 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654f9f4f-f2c8-48f1-b032-3fc23be1048a None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:28.070444 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654f9f4f-f2c8-48f1-b032-3fc23be1048a None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:28.070923 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654f9f4f-f2c8-48f1-b032-3fc23be1048a None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:28.076250 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 424/840] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 04:59:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 18 04:59:28.077992 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7366b8-000a-4166-a27e-e6846bab211f None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:28.078488 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7366b8-000a-4166-a27e-e6846bab211f None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:28.078981 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7366b8-000a-4166-a27e-e6846bab211f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:28.084378 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa18ace-3d7f-4315-a139-30f38c81e142 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:28.097312 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:28.098240 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-1e7366b8-000a-4166-a27e-e6846bab211f None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 04:59:28.100532 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-1e7366b8-000a-4166-a27e-e6846bab211f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 04:59:28.127183 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189047b9-67b5-40db-bd90-9925a3804a0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:28.127878 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189047b9-67b5-40db-bd90-9925a3804a0f None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:28.128151 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189047b9-67b5-40db-bd90-9925a3804a0f None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:28.128430 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189047b9-67b5-40db-bd90-9925a3804a0f None admin] PATH_INFO: `/v3/users` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:28.129474 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189047b9-67b5-40db-bd90-9925a3804a0f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:28.129798 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189047b9-67b5-40db-bd90-9925a3804a0f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:28.132673 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189047b9-67b5-40db-bd90-9925a3804a0f None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:28.157411 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 417/841] 10.222.0.23 () {56 vars in 1135 bytes} [Sat Jun 18 04:59:27 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:28.162653 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189047b9-67b5-40db-bd90-9925a3804a0f None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:28.183497 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 425/842] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 04:59:28 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 18 04:59:28.201693 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7366b8-000a-4166-a27e-e6846bab211f None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:28.258838 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:28.292101 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189047b9-67b5-40db-bd90-9925a3804a0f None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:28.311371 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7366b8-000a-4166-a27e-e6846bab211f None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:28.328360 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26556ffb-3505-4136-aac5-7e60be646695 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:28.329119 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26556ffb-3505-4136-aac5-7e60be646695 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:28.329523 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26556ffb-3505-4136-aac5-7e60be646695 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:28.329870 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26556ffb-3505-4136-aac5-7e60be646695 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:28.355904 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0a3dce-fbeb-405a-ad7d-3220aef87ffd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:28.357765 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0a3dce-fbeb-405a-ad7d-3220aef87ffd None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:28.358362 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0a3dce-fbeb-405a-ad7d-3220aef87ffd None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:28.358900 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0a3dce-fbeb-405a-ad7d-3220aef87ffd None admin] PATH_INFO: `/v3/roles/service` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:28.361272 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:28.374464 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0a3dce-fbeb-405a-ad7d-3220aef87ffd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 18 04:59:28.383879 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-8b0a3dce-fbeb-405a-ad7d-3220aef87ffd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 18 04:59:28.383879 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:28.383879 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:59:28.383879 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:59:28.383879 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:59:28.383879 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:59:28.383879 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:59:28.383879 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:59:28.383879 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:59:28.383879 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:59:28.383879 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:59:28.383879 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:59:28.383879 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 18 04:59:28.383879 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 18 04:59:28.383879 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 18 04:59:28.383879 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 18 04:59:28.385104 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 18 04:59:28.385104 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 18 04:59:28.385104 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 18 04:59:28.385104 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 18 04:59:28.385104 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:59:28.385104 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:59:28.385104 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:59:28.385104 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:59:28.385104 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:59:28.385104 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:59:28.385104 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:59:28.385104 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jun 18 04:59:28.385104 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:59:28.385104 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:59:28.385104 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:59:28.385104 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:59:28.386038 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:59:28.386038 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:59:28.386038 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:59:28.386038 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:59:28.386038 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 18 04:59:28.386038 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 18 04:59:28.386038 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 18 04:59:28.386038 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 18 04:59:28.386038 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 18 04:59:28.386038 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 18 04:59:28.386038 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 18 04:59:28.386038 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 04:59:28.387458 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 426/843] 10.222.0.23 () {56 vars in 1102 bytes} [Sat Jun 18 04:59:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:59:28.398716 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0a3dce-fbeb-405a-ad7d-3220aef87ffd None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:28.422551 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26556ffb-3505-4136-aac5-7e60be646695 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:28.424388 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26556ffb-3505-4136-aac5-7e60be646695 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:28.515149 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0a3dce-fbeb-405a-ad7d-3220aef87ffd None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:28.533278 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:28.563304 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 418/844] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Jun 18 04:59:28 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 18 04:59:28.564081 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fcf44c-c5fb-418b-8edc-36e9bf7d24c8 None None] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:28.564257 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a50904-1d69-4cac-a46d-58bf4afb2e14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:28.564398 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fcf44c-c5fb-418b-8edc-36e9bf7d24c8 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:28.564637 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fcf44c-c5fb-418b-8edc-36e9bf7d24c8 None None] PATH_INFO: `/` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:28.564940 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a50904-1d69-4cac-a46d-58bf4afb2e14 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:28.565225 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a50904-1d69-4cac-a46d-58bf4afb2e14 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:28.565536 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a50904-1d69-4cac-a46d-58bf4afb2e14 None admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:28.567400 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 419/845] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 04:59:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 18 04:59:28.567541 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a50904-1d69-4cac-a46d-58bf4afb2e14 None admin] RBAC: Adding query filter params (name=service) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:28.567541 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a50904-1d69-4cac-a46d-58bf4afb2e14 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:28.567687 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fcf44c-c5fb-418b-8edc-36e9bf7d24c8 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:28.569611 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a50904-1d69-4cac-a46d-58bf4afb2e14 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:28.577328 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 427/846] 10.222.0.23 () {56 vars in 1103 bytes} [Sat Jun 18 04:59:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 18 04:59:28.578828 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5e5517-39e6-496c-b26e-cfeabe80b0d4 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:28.579310 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5e5517-39e6-496c-b26e-cfeabe80b0d4 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:28.579781 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5e5517-39e6-496c-b26e-cfeabe80b0d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:28.596689 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-9a5e5517-39e6-496c-b26e-cfeabe80b0d4 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 04:59:28.597926 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-9a5e5517-39e6-496c-b26e-cfeabe80b0d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 04:59:28.636498 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:28.669559 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 428/847] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 04:59:28 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 18 04:59:28.669559 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5e5517-39e6-496c-b26e-cfeabe80b0d4 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:28.691844 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fcf44c-c5fb-418b-8edc-36e9bf7d24c8 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:28.729751 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2484a090-e85a-4fe2-b542-27b527370c9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:28.731300 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2484a090-e85a-4fe2-b542-27b527370c9a None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:28.731763 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2484a090-e85a-4fe2-b542-27b527370c9a None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:28.732216 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2484a090-e85a-4fe2-b542-27b527370c9a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:28.746833 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2484a090-e85a-4fe2-b542-27b527370c9a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:28.748276 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2484a090-e85a-4fe2-b542-27b527370c9a None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:28.768788 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application [None req-2484a090-e85a-4fe2-b542-27b527370c9a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 04:59:28.768788 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:28.768788 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:59:28.768788 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:59:28.768788 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:59:28.768788 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:59:28.768788 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:59:28.768788 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:59:28.768788 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:59:28.768788 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:59:28.768788 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:59:28.768788 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:59:28.768788 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 18 04:59:28.768788 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 18 04:59:28.768788 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 18 04:59:28.770008 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 18 04:59:28.770008 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:59:28.770008 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:59:28.770008 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:59:28.770008 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:59:28.770008 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:59:28.770008 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:59:28.770008 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:59:28.770008 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application with Lock( Jun 18 04:59:28.770008 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:59:28.770008 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:59:28.770008 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:59:28.770008 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:59:28.770008 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:59:28.770008 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:59:28.770008 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:59:28.770008 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:59:28.772354 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 18 04:59:28.772354 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 18 04:59:28.772354 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 04:59:28.772354 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 04:59:28.772354 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 420/848] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Jun 18 04:59:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:59:28.774030 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2484a090-e85a-4fe2-b542-27b527370c9a None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:28.785427 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5e5517-39e6-496c-b26e-cfeabe80b0d4 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:28.793179 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:28.820825 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb6ae48-7ee9-4fab-84fd-0528ecdffda1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:28.821980 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb6ae48-7ee9-4fab-84fd-0528ecdffda1 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:28.822516 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb6ae48-7ee9-4fab-84fd-0528ecdffda1 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:28.823156 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb6ae48-7ee9-4fab-84fd-0528ecdffda1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:28.834505 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb6ae48-7ee9-4fab-84fd-0528ecdffda1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:28.835968 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb6ae48-7ee9-4fab-84fd-0528ecdffda1 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:28.853907 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-1eb6ae48-7ee9-4fab-84fd-0528ecdffda1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 04:59:28.853907 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:28.853907 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:59:28.853907 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:59:28.853907 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:59:28.853907 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:59:28.853907 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:59:28.853907 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:59:28.853907 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:59:28.853907 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:59:28.853907 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:59:28.853907 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:59:28.853907 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 18 04:59:28.853907 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 18 04:59:28.853907 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 18 04:59:28.854825 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 18 04:59:28.854825 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:59:28.854825 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:59:28.854825 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:59:28.854825 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:59:28.854825 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:59:28.854825 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:59:28.854825 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:59:28.854825 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jun 18 04:59:28.854825 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:59:28.854825 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:59:28.854825 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:59:28.854825 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:59:28.854825 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:59:28.854825 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:59:28.854825 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:59:28.854825 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:59:28.855716 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 18 04:59:28.855716 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 18 04:59:28.855716 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 04:59:28.855716 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 04:59:28.855716 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 429/849] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Jun 18 04:59:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:59:28.855995 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb6ae48-7ee9-4fab-84fd-0528ecdffda1 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:28.897133 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2484a090-e85a-4fe2-b542-27b527370c9a None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:28.911802 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:28.933065 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312fee43-bdf5-4e2a-8859-e20fa65dc131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:28.933887 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312fee43-bdf5-4e2a-8859-e20fa65dc131 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:28.934233 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312fee43-bdf5-4e2a-8859-e20fa65dc131 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:28.934504 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312fee43-bdf5-4e2a-8859-e20fa65dc131 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:28.935148 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312fee43-bdf5-4e2a-8859-e20fa65dc131 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:28.936501 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312fee43-bdf5-4e2a-8859-e20fa65dc131 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:28.949365 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application [None req-312fee43-bdf5-4e2a-8859-e20fa65dc131 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 18 04:59:28.949365 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:28.949365 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 18 04:59:28.949365 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 18 04:59:28.949365 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 18 04:59:28.949365 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 18 04:59:28.949365 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 18 04:59:28.949365 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 18 04:59:28.949365 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 18 04:59:28.949365 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 04:59:28.949365 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 18 04:59:28.949365 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 04:59:28.949365 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:28.949365 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:59:28.949365 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:59:28.949365 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:59:28.949365 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:59:28.950415 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:59:28.950415 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:59:28.950415 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:59:28.950415 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:59:28.950415 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:59:28.950415 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:59:28.950415 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 18 04:59:28.950415 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 18 04:59:28.950415 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 18 04:59:28.950415 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 18 04:59:28.950415 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:59:28.950415 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:59:28.950415 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:59:28.950415 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:59:28.950415 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:59:28.950415 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:59:28.951269 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:59:28.951269 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application with Lock( Jun 18 04:59:28.951269 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:59:28.951269 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:59:28.951269 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:59:28.951269 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:59:28.951269 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:59:28.951269 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:59:28.951269 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:59:28.951269 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:59:28.951269 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 18 04:59:28.951269 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 18 04:59:28.951269 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 18 04:59:28.951269 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 04:59:28.951269 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 421/850] 10.222.0.23 () {56 vars in 1114 bytes} [Sat Jun 18 04:59:28 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:59:28.952184 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312fee43-bdf5-4e2a-8859-e20fa65dc131 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:28.970389 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb6ae48-7ee9-4fab-84fd-0528ecdffda1 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:29.006746 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71782e7-8033-42a1-936d-aa9601510b24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:29.007581 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71782e7-8033-42a1-936d-aa9601510b24 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:29.007866 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71782e7-8033-42a1-936d-aa9601510b24 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:29.008142 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71782e7-8033-42a1-936d-aa9601510b24 None admin] PATH_INFO: `/v3/domains` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:29.008909 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71782e7-8033-42a1-936d-aa9601510b24 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:29.009161 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71782e7-8033-42a1-936d-aa9601510b24 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:29.010576 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71782e7-8033-42a1-936d-aa9601510b24 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:29.026962 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 430/851] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Jun 18 04:59:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:29.026962 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71782e7-8033-42a1-936d-aa9601510b24 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:29.047716 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:29.073916 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312fee43-bdf5-4e2a-8859-e20fa65dc131 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:29.110935 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499a3165-180e-4d3e-a3c7-fb44cf770e12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:29.111778 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499a3165-180e-4d3e-a3c7-fb44cf770e12 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:29.112275 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499a3165-180e-4d3e-a3c7-fb44cf770e12 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:29.112502 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499a3165-180e-4d3e-a3c7-fb44cf770e12 None admin] PATH_INFO: `/v3/domains` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:29.113296 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499a3165-180e-4d3e-a3c7-fb44cf770e12 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:29.113701 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499a3165-180e-4d3e-a3c7-fb44cf770e12 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:29.115697 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499a3165-180e-4d3e-a3c7-fb44cf770e12 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:29.128437 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 422/852] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Jun 18 04:59:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:29.132613 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71782e7-8033-42a1-936d-aa9601510b24 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:29.133722 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499a3165-180e-4d3e-a3c7-fb44cf770e12 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:29.168594 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32249815-a6d8-41ae-a1a0-7e576c037a4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:29.169256 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32249815-a6d8-41ae-a1a0-7e576c037a4f None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:29.169579 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32249815-a6d8-41ae-a1a0-7e576c037a4f None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:29.169856 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32249815-a6d8-41ae-a1a0-7e576c037a4f None admin] PATH_INFO: `/v3/services` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:29.170614 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32249815-a6d8-41ae-a1a0-7e576c037a4f None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:29.170923 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32249815-a6d8-41ae-a1a0-7e576c037a4f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:29.172288 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32249815-a6d8-41ae-a1a0-7e576c037a4f None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:29.173602 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:29.176975 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 431/853] 10.222.0.23 () {56 vars in 1114 bytes} [Sat Jun 18 04:59:29 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:29.178323 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32249815-a6d8-41ae-a1a0-7e576c037a4f None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:29.241801 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499a3165-180e-4d3e-a3c7-fb44cf770e12 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:29.269292 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:29.277145 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1588d9-f1e8-43c2-8775-830a4796e0bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:29.277975 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1588d9-f1e8-43c2-8775-830a4796e0bd None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:29.278326 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1588d9-f1e8-43c2-8775-830a4796e0bd None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:29.278664 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1588d9-f1e8-43c2-8775-830a4796e0bd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:29.285587 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1588d9-f1e8-43c2-8775-830a4796e0bd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:29.287226 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1588d9-f1e8-43c2-8775-830a4796e0bd None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:29.289244 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32249815-a6d8-41ae-a1a0-7e576c037a4f None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:29.300832 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application [None req-1d1588d9-f1e8-43c2-8775-830a4796e0bd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 18 04:59:29.300832 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:29.300832 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:59:29.300832 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:59:29.300832 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:59:29.300832 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:59:29.300832 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:59:29.300832 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:59:29.300832 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:59:29.300832 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:59:29.300832 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:59:29.300832 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:59:29.300832 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 18 04:59:29.300832 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 18 04:59:29.300832 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 18 04:59:29.301981 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 18 04:59:29.301981 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:59:29.301981 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:59:29.301981 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:59:29.301981 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:59:29.301981 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:59:29.301981 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:59:29.301981 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:59:29.301981 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application with Lock( Jun 18 04:59:29.301981 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:59:29.301981 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:59:29.301981 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:59:29.301981 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:59:29.301981 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:59:29.301981 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:59:29.301981 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:59:29.301981 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:59:29.303672 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 18 04:59:29.303672 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 18 04:59:29.303672 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 18 04:59:29.303672 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 18 04:59:29.303672 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 18 04:59:29.303672 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 18 04:59:29.303672 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 18 04:59:29.303672 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 04:59:29.304987 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 423/854] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Jun 18 04:59:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:59:29.305527 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1588d9-f1e8-43c2-8775-830a4796e0bd None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:29.333392 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea76732-e916-49a2-a82d-ad7c83101ccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:29.334825 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea76732-e916-49a2-a82d-ad7c83101ccb None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:29.335138 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea76732-e916-49a2-a82d-ad7c83101ccb None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:29.335413 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea76732-e916-49a2-a82d-ad7c83101ccb None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:29.343239 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea76732-e916-49a2-a82d-ad7c83101ccb None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:29.344686 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea76732-e916-49a2-a82d-ad7c83101ccb None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:29.359997 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-5ea76732-e916-49a2-a82d-ad7c83101ccb None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 18 04:59:29.359997 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:29.359997 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:59:29.359997 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:59:29.359997 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:59:29.359997 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:59:29.359997 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:59:29.359997 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:59:29.359997 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:59:29.359997 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:59:29.359997 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:59:29.359997 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:59:29.359997 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 18 04:59:29.359997 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 18 04:59:29.359997 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 18 04:59:29.359997 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 18 04:59:29.361837 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:59:29.361837 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:59:29.361837 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 18 04:59:29.361837 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 18 04:59:29.361837 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 18 04:59:29.361837 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 18 04:59:29.361837 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:59:29.361837 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:59:29.361837 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:59:29.361837 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:59:29.361837 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:59:29.361837 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jun 18 04:59:29.361837 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:59:29.361837 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:59:29.361837 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:59:29.361837 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:59:29.361837 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:59:29.361837 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:59:29.362988 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:59:29.362988 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:59:29.362988 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 18 04:59:29.362988 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 18 04:59:29.362988 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 18 04:59:29.362988 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 18 04:59:29.362988 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 18 04:59:29.362988 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 18 04:59:29.362988 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 18 04:59:29.362988 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 04:59:29.362988 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 432/855] 10.222.0.23 () {56 vars in 1117 bytes} [Sat Jun 18 04:59:29 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:59:29.362988 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea76732-e916-49a2-a82d-ad7c83101ccb None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:29.417890 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:29.423623 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1588d9-f1e8-43c2-8775-830a4796e0bd None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:29.456609 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9644d2-c09b-488a-8224-530baab44746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:29.457266 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9644d2-c09b-488a-8224-530baab44746 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:29.457591 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9644d2-c09b-488a-8224-530baab44746 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:29.457871 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9644d2-c09b-488a-8224-530baab44746 None admin] PATH_INFO: `/v3/services` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:29.459136 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9644d2-c09b-488a-8224-530baab44746 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:29.459476 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9644d2-c09b-488a-8224-530baab44746 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:29.460829 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9644d2-c09b-488a-8224-530baab44746 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:29.465280 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 424/856] 10.222.0.23 () {56 vars in 1114 bytes} [Sat Jun 18 04:59:29 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:29.466512 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1c8ad7-a6e4-4f87-845b-fda727a08a0e None None] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:29.466795 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1c8ad7-a6e4-4f87-845b-fda727a08a0e None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:29.467068 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1c8ad7-a6e4-4f87-845b-fda727a08a0e None None] PATH_INFO: `/` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:29.467653 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea76732-e916-49a2-a82d-ad7c83101ccb None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:29.468532 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 425/857] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 04:59:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 04:59:29.469090 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1c8ad7-a6e4-4f87-845b-fda727a08a0e None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:29.513273 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f6dd21-ab26-4a3d-b97f-d8ff4b3e00a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:29.514279 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f6dd21-ab26-4a3d-b97f-d8ff4b3e00a2 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:29.514729 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f6dd21-ab26-4a3d-b97f-d8ff4b3e00a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:29.515006 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f6dd21-ab26-4a3d-b97f-d8ff4b3e00a2 None admin] PATH_INFO: `/v3/projects` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:29.515833 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f6dd21-ab26-4a3d-b97f-d8ff4b3e00a2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:29.516146 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f6dd21-ab26-4a3d-b97f-d8ff4b3e00a2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:29.517687 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f6dd21-ab26-4a3d-b97f-d8ff4b3e00a2 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:29.537368 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 433/858] 10.222.0.23 () {56 vars in 1148 bytes} [Sat Jun 18 04:59:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:29.537368 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef20acd-8b20-4368-a375-34ce5c4f4c0b None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:29.537368 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef20acd-8b20-4368-a375-34ce5c4f4c0b None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:29.537368 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef20acd-8b20-4368-a375-34ce5c4f4c0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:29.541958 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:29.555122 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-fef20acd-8b20-4368-a375-34ce5c4f4c0b None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 04:59:29.556461 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-fef20acd-8b20-4368-a375-34ce5c4f4c0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 04:59:29.592005 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1c8ad7-a6e4-4f87-845b-fda727a08a0e None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:29.631718 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2b39d3-68dd-4df2-9c0e-245190faac3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:29.632266 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 434/859] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 04:59:29 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 18 04:59:29.632467 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2b39d3-68dd-4df2-9c0e-245190faac3e None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:29.632755 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2b39d3-68dd-4df2-9c0e-245190faac3e None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:29.633026 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2b39d3-68dd-4df2-9c0e-245190faac3e None admin] PATH_INFO: `/v3/users` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:29.633123 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef20acd-8b20-4368-a375-34ce5c4f4c0b None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:29.633979 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2b39d3-68dd-4df2-9c0e-245190faac3e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:29.637404 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2b39d3-68dd-4df2-9c0e-245190faac3e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:29.637404 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2b39d3-68dd-4df2-9c0e-245190faac3e None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:29.656940 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 426/860] 10.222.0.23 () {56 vars in 1149 bytes} [Sat Jun 18 04:59:29 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:29.658106 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2b39d3-68dd-4df2-9c0e-245190faac3e None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:29.682721 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:29.747793 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef20acd-8b20-4368-a375-34ce5c4f4c0b None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:29.777932 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:29.778699 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2b39d3-68dd-4df2-9c0e-245190faac3e None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:29.783285 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8393c90-be9c-4895-93ea-c9f54da720e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:29.783937 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8393c90-be9c-4895-93ea-c9f54da720e1 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:29.784222 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8393c90-be9c-4895-93ea-c9f54da720e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:29.784495 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8393c90-be9c-4895-93ea-c9f54da720e1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:29.785333 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8393c90-be9c-4895-93ea-c9f54da720e1 None admin] RBAC: Adding query filter params (role.id=21c65f2c51bd4dd283d13507d2714467, scope.project.id=c95f8f3084c5472cb8364b7c57207394, user.id=16b49e866398400580ae48aa5a96fc04) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:29.785677 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8393c90-be9c-4895-93ea-c9f54da720e1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:29.787810 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8393c90-be9c-4895-93ea-c9f54da720e1 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:29.801549 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 435/861] 10.222.0.23 () {56 vars in 1374 bytes} [Sat Jun 18 04:59:29 2022] GET /identity/v3/role_assignments?user.id=16b49e866398400580ae48aa5a96fc04&scope.project.id=c95f8f3084c5472cb8364b7c57207394&role.id=21c65f2c51bd4dd283d13507d2714467 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:29.801549 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8393c90-be9c-4895-93ea-c9f54da720e1 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:29.832210 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262e6596-f0c1-47bc-813b-cd4fe5de647c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:29.833135 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262e6596-f0c1-47bc-813b-cd4fe5de647c None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:29.834245 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262e6596-f0c1-47bc-813b-cd4fe5de647c None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:29.835582 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262e6596-f0c1-47bc-813b-cd4fe5de647c None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:29.836401 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262e6596-f0c1-47bc-813b-cd4fe5de647c None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:29.837912 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262e6596-f0c1-47bc-813b-cd4fe5de647c None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:29.855053 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application [None req-262e6596-f0c1-47bc-813b-cd4fe5de647c None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 18 04:59:29.855053 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:29.855053 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 18 04:59:29.855053 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 18 04:59:29.855053 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 18 04:59:29.855053 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 18 04:59:29.855053 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 18 04:59:29.855053 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 18 04:59:29.855053 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 18 04:59:29.855053 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 04:59:29.855053 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 18 04:59:29.855053 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 04:59:29.855053 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:29.855053 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:59:29.855053 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:59:29.855053 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:59:29.856238 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:59:29.856238 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:59:29.856238 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:59:29.856238 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:59:29.856238 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:59:29.856238 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:59:29.856238 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:59:29.856238 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 18 04:59:29.856238 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 18 04:59:29.856238 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 18 04:59:29.856238 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 18 04:59:29.856238 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:59:29.856238 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:59:29.856238 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:59:29.856238 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:59:29.856238 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:59:29.857063 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:59:29.857063 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:59:29.857063 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application with Lock( Jun 18 04:59:29.857063 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:59:29.857063 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:59:29.857063 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:59:29.857063 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:59:29.857063 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:59:29.857063 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:59:29.857063 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:59:29.857063 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:59:29.857063 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 18 04:59:29.857063 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 18 04:59:29.857063 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 18 04:59:29.857063 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 04:59:29.857890 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 427/862] 10.222.0.23 () {56 vars in 1132 bytes} [Sat Jun 18 04:59:29 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:59:29.863897 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262e6596-f0c1-47bc-813b-cd4fe5de647c None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:29.916785 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8393c90-be9c-4895-93ea-c9f54da720e1 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:29.956772 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d127f35b-dc55-4a59-947d-852658a538aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:29.957525 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d127f35b-dc55-4a59-947d-852658a538aa None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:29.957810 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d127f35b-dc55-4a59-947d-852658a538aa None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:29.958089 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d127f35b-dc55-4a59-947d-852658a538aa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:29.984366 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:30.017427 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262e6596-f0c1-47bc-813b-cd4fe5de647c None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:30.043477 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:30.049452 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d127f35b-dc55-4a59-947d-852658a538aa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:30.050946 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d127f35b-dc55-4a59-947d-852658a538aa None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:30.059251 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f65a5ec-ade3-4408-88f6-faf31fc1d465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:30.059926 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f65a5ec-ade3-4408-88f6-faf31fc1d465 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:30.060506 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f65a5ec-ade3-4408-88f6-faf31fc1d465 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:30.061158 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f65a5ec-ade3-4408-88f6-faf31fc1d465 None admin] PATH_INFO: `/v3/services` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:30.061511 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f65a5ec-ade3-4408-88f6-faf31fc1d465 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:30.061823 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f65a5ec-ade3-4408-88f6-faf31fc1d465 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:30.063182 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f65a5ec-ade3-4408-88f6-faf31fc1d465 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:30.069106 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 428/863] 10.222.0.23 () {56 vars in 1126 bytes} [Sat Jun 18 04:59:29 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:30.074735 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f65a5ec-ade3-4408-88f6-faf31fc1d465 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:30.169602 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 436/864] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Jun 18 04:59:29 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 18 04:59:30.170821 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d127f35b-dc55-4a59-947d-852658a538aa None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:30.217607 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f65a5ec-ade3-4408-88f6-faf31fc1d465 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:30.261211 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8af406e-7939-4141-bbde-a95b851f5760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:30.261903 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8af406e-7939-4141-bbde-a95b851f5760 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:30.262189 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8af406e-7939-4141-bbde-a95b851f5760 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:30.262465 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8af406e-7939-4141-bbde-a95b851f5760 None admin] PATH_INFO: `/v3/services` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:30.263220 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8af406e-7939-4141-bbde-a95b851f5760 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:30.263532 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8af406e-7939-4141-bbde-a95b851f5760 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:30.264894 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8af406e-7939-4141-bbde-a95b851f5760 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:30.269957 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 429/865] 10.222.0.23 () {56 vars in 1126 bytes} [Sat Jun 18 04:59:30 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 18 04:59:30.287936 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d127f35b-dc55-4a59-947d-852658a538aa None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:30.305676 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:30.328214 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dac512-d426-40c8-9952-11809a30a51b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:30.329011 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dac512-d426-40c8-9952-11809a30a51b None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:30.329295 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dac512-d426-40c8-9952-11809a30a51b None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:30.329598 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dac512-d426-40c8-9952-11809a30a51b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:30.341833 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dac512-d426-40c8-9952-11809a30a51b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:30.343330 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dac512-d426-40c8-9952-11809a30a51b None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:30.363581 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-87dac512-d426-40c8-9952-11809a30a51b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 04:59:30.363581 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:30.363581 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:59:30.363581 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:59:30.363581 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:59:30.363581 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:59:30.363581 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:59:30.363581 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:59:30.363581 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:59:30.363581 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:59:30.363581 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:59:30.363581 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:59:30.363581 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 18 04:59:30.363581 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 18 04:59:30.363581 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 18 04:59:30.365073 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 18 04:59:30.365073 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:59:30.365073 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:59:30.365073 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:59:30.365073 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:59:30.365073 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:59:30.365073 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:59:30.365073 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:59:30.365073 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jun 18 04:59:30.365073 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:59:30.365073 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:59:30.365073 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:59:30.365073 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:59:30.365073 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:59:30.365073 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:59:30.365073 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:59:30.365073 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:59:30.366257 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 18 04:59:30.366257 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 18 04:59:30.366257 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 04:59:30.366257 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 04:59:30.366257 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 437/866] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Jun 18 04:59:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:59:30.372949 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8af406e-7939-4141-bbde-a95b851f5760 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:30.388611 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:30.487785 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8af406e-7939-4141-bbde-a95b851f5760 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:30.522775 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204818c8-6fe6-4744-98fe-ecd834ff000d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:30.523506 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204818c8-6fe6-4744-98fe-ecd834ff000d None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:30.523789 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204818c8-6fe6-4744-98fe-ecd834ff000d None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:30.524101 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204818c8-6fe6-4744-98fe-ecd834ff000d None admin] PATH_INFO: `/v3/domains` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:30.524833 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204818c8-6fe6-4744-98fe-ecd834ff000d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:30.525144 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204818c8-6fe6-4744-98fe-ecd834ff000d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:30.526543 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204818c8-6fe6-4744-98fe-ecd834ff000d None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:30.539659 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 430/867] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Jun 18 04:59:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:30.545168 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dac512-d426-40c8-9952-11809a30a51b None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:30.656374 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dac512-d426-40c8-9952-11809a30a51b None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:30.694289 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4149e7b-59d3-4166-aefd-b26ba78088e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:30.695443 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4149e7b-59d3-4166-aefd-b26ba78088e1 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:30.695970 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4149e7b-59d3-4166-aefd-b26ba78088e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:30.696452 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4149e7b-59d3-4166-aefd-b26ba78088e1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:30.703835 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4149e7b-59d3-4166-aefd-b26ba78088e1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:30.705588 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4149e7b-59d3-4166-aefd-b26ba78088e1 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:30.719990 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-c4149e7b-59d3-4166-aefd-b26ba78088e1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 18 04:59:30.719990 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:30.719990 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:59:30.719990 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:59:30.719990 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:59:30.719990 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:59:30.719990 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:59:30.719990 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:59:30.719990 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:59:30.719990 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:59:30.719990 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:59:30.719990 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:59:30.719990 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 18 04:59:30.719990 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 18 04:59:30.719990 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 18 04:59:30.720877 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 18 04:59:30.720877 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:59:30.720877 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:59:30.720877 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:59:30.720877 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:59:30.720877 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:59:30.720877 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:59:30.720877 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:59:30.720877 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jun 18 04:59:30.720877 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:59:30.720877 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:59:30.720877 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:59:30.720877 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:59:30.720877 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:59:30.720877 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:59:30.720877 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:59:30.720877 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:59:30.721802 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 18 04:59:30.721802 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 18 04:59:30.721802 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 18 04:59:30.721802 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 18 04:59:30.721802 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 18 04:59:30.721802 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 18 04:59:30.721802 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 18 04:59:30.721802 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 04:59:30.724200 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 438/868] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Jun 18 04:59:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:59:30.730630 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204818c8-6fe6-4744-98fe-ecd834ff000d None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:30.810683 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:30.840973 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204818c8-6fe6-4744-98fe-ecd834ff000d None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:30.875881 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fac4ad0-68eb-4006-831e-c7bd26521107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:30.876539 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fac4ad0-68eb-4006-831e-c7bd26521107 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:30.876827 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fac4ad0-68eb-4006-831e-c7bd26521107 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:30.877104 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fac4ad0-68eb-4006-831e-c7bd26521107 None admin] PATH_INFO: `/v3/projects` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:30.877923 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fac4ad0-68eb-4006-831e-c7bd26521107 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:30.878236 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fac4ad0-68eb-4006-831e-c7bd26521107 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:30.879652 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fac4ad0-68eb-4006-831e-c7bd26521107 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:30.892486 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 431/869] 10.222.0.23 () {56 vars in 1148 bytes} [Sat Jun 18 04:59:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:30.898404 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4149e7b-59d3-4166-aefd-b26ba78088e1 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:30.911211 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:31.009806 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4149e7b-59d3-4166-aefd-b26ba78088e1 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:31.046326 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b69b58-f95d-4540-b586-1f363a596697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:31.047393 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b69b58-f95d-4540-b586-1f363a596697 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:31.047914 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b69b58-f95d-4540-b586-1f363a596697 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:31.048398 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b69b58-f95d-4540-b586-1f363a596697 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:31.049496 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b69b58-f95d-4540-b586-1f363a596697 None admin] RBAC: Adding query filter params (role.id=5ec7b68b793d4a54abd30460961346b9, scope.project.id=c95f8f3084c5472cb8364b7c57207394, user.id=e817fafd0c1b4bf9944481cf116f1260) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:31.050022 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b69b58-f95d-4540-b586-1f363a596697 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:31.051697 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b69b58-f95d-4540-b586-1f363a596697 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:31.059895 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 439/870] 10.222.0.23 () {56 vars in 1374 bytes} [Sat Jun 18 04:59:30 2022] GET /identity/v3/role_assignments?user.id=e817fafd0c1b4bf9944481cf116f1260&scope.project.id=c95f8f3084c5472cb8364b7c57207394&role.id=5ec7b68b793d4a54abd30460961346b9 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:31.562854 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1919ef6d-432e-4e2a-875d-a1be4c7c1f16 None None] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:31.563359 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1919ef6d-432e-4e2a-875d-a1be4c7c1f16 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:31.563433 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1919ef6d-432e-4e2a-875d-a1be4c7c1f16 None None] PATH_INFO: `/` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:31.565063 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 432/871] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 04:59:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 04:59:31.571206 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a079921a-7bc6-425a-acc6-d26a68a36a69 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:31.571668 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a079921a-7bc6-425a-acc6-d26a68a36a69 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:31.571812 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a079921a-7bc6-425a-acc6-d26a68a36a69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:31.589835 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-a079921a-7bc6-425a-acc6-d26a68a36a69 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 04:59:31.591038 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-a079921a-7bc6-425a-acc6-d26a68a36a69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 04:59:31.669834 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 440/872] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 04:59:31 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 18 04:59:31.692780 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1919ef6d-432e-4e2a-875d-a1be4c7c1f16 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:31.805656 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1919ef6d-432e-4e2a-875d-a1be4c7c1f16 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:31.836278 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:31.842735 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05617c71-85fb-4563-ba86-7179fd6d6c64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:31.843153 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05617c71-85fb-4563-ba86-7179fd6d6c64 None admin] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:31.843439 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05617c71-85fb-4563-ba86-7179fd6d6c64 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:31.843716 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05617c71-85fb-4563-ba86-7179fd6d6c64 None admin] PATH_INFO: `/v3/services` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:31.844462 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05617c71-85fb-4563-ba86-7179fd6d6c64 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:31.845915 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05617c71-85fb-4563-ba86-7179fd6d6c64 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:31.929661 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 433/873] 10.222.0.23 () {60 vars in 1141 bytes} [Sat Jun 18 04:59:31 2022] POST /identity/v3/services => generated 251 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 04:59:32.013921 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99004f57-8fd8-4bce-918a-5c43614c34ea None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:32.014259 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99004f57-8fd8-4bce-918a-5c43614c34ea None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:32.014494 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99004f57-8fd8-4bce-918a-5c43614c34ea None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:32.016279 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 441/874] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 04:59:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 04:59:32.022957 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcce8806-470b-4c17-8c50-03c6beb5fdc7 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:32.023291 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcce8806-470b-4c17-8c50-03c6beb5fdc7 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:32.023517 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcce8806-470b-4c17-8c50-03c6beb5fdc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:32.042268 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-fcce8806-470b-4c17-8c50-03c6beb5fdc7 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 04:59:32.043511 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-fcce8806-470b-4c17-8c50-03c6beb5fdc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 04:59:32.131216 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 434/875] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 04:59:32 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 18 04:59:32.156357 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99004f57-8fd8-4bce-918a-5c43614c34ea None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:32.159783 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:32.269361 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99004f57-8fd8-4bce-918a-5c43614c34ea None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:32.309603 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82095939-16fd-4647-b570-33e74058604a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:32.310793 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82095939-16fd-4647-b570-33e74058604a None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:32.311283 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82095939-16fd-4647-b570-33e74058604a None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:32.311927 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82095939-16fd-4647-b570-33e74058604a None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:32.312796 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82095939-16fd-4647-b570-33e74058604a None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:32.314508 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82095939-16fd-4647-b570-33e74058604a None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:32.329855 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-82095939-16fd-4647-b570-33e74058604a None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 18 04:59:32.329855 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:32.329855 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 18 04:59:32.329855 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 18 04:59:32.329855 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 18 04:59:32.329855 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 18 04:59:32.329855 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 18 04:59:32.329855 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 18 04:59:32.329855 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 18 04:59:32.329855 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 04:59:32.329855 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 18 04:59:32.329855 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 04:59:32.329855 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:32.329855 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:59:32.329855 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:59:32.329855 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:59:32.329855 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:59:32.330885 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:59:32.330885 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:59:32.330885 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:59:32.330885 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:59:32.330885 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:59:32.330885 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:59:32.330885 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 18 04:59:32.330885 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 18 04:59:32.330885 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 18 04:59:32.330885 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 18 04:59:32.330885 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:59:32.330885 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:59:32.330885 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:59:32.330885 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:59:32.330885 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:59:32.330885 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:59:32.331727 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:59:32.331727 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jun 18 04:59:32.331727 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:59:32.331727 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:59:32.331727 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:59:32.331727 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:59:32.331727 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:59:32.331727 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:59:32.331727 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:59:32.331727 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:59:32.331727 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 18 04:59:32.331727 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 18 04:59:32.331727 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 18 04:59:32.331727 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 04:59:32.335442 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 442/876] 10.222.0.23 () {56 vars in 1126 bytes} [Sat Jun 18 04:59:32 2022] GET /identity/v3/services/object-store => generated 93 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:59:32.345435 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcce8806-470b-4c17-8c50-03c6beb5fdc7 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:32.452138 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcce8806-470b-4c17-8c50-03c6beb5fdc7 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:32.470859 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:32.487656 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c55f7a-d2ce-4799-8397-a2993a9d4cc2 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:32.497626 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c55f7a-d2ce-4799-8397-a2993a9d4cc2 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:32.497626 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c55f7a-d2ce-4799-8397-a2993a9d4cc2 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:32.497626 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 443/877] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 04:59:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 04:59:32.497626 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003cbf17-d736-4aa5-b968-8fc3d7534dc3 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:32.497956 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c34f94-fe60-49f6-b29d-f2440232c7d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:32.498049 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003cbf17-d736-4aa5-b968-8fc3d7534dc3 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:32.498049 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003cbf17-d736-4aa5-b968-8fc3d7534dc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:32.498187 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c34f94-fe60-49f6-b29d-f2440232c7d8 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:32.498187 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c34f94-fe60-49f6-b29d-f2440232c7d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:32.498187 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c34f94-fe60-49f6-b29d-f2440232c7d8 None admin] PATH_INFO: `/v3/services` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:32.498187 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c34f94-fe60-49f6-b29d-f2440232c7d8 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:32.498761 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c34f94-fe60-49f6-b29d-f2440232c7d8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:32.500275 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c34f94-fe60-49f6-b29d-f2440232c7d8 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:32.510474 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 435/878] 10.222.0.23 () {56 vars in 1122 bytes} [Sat Jun 18 04:59:32 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:32.511243 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c34f94-fe60-49f6-b29d-f2440232c7d8 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:32.516664 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-003cbf17-d736-4aa5-b968-8fc3d7534dc3 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 04:59:32.517886 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-003cbf17-d736-4aa5-b968-8fc3d7534dc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 04:59:32.597473 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:32.612322 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 444/879] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 04:59:32 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 18 04:59:32.633528 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003cbf17-d736-4aa5-b968-8fc3d7534dc3 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:32.642348 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c34f94-fe60-49f6-b29d-f2440232c7d8 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:32.690002 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc6e25c-398e-436b-8426-3360255ab479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:32.690664 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc6e25c-398e-436b-8426-3360255ab479 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:32.690947 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc6e25c-398e-436b-8426-3360255ab479 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:32.691222 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc6e25c-398e-436b-8426-3360255ab479 None admin] PATH_INFO: `/v3/services` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:32.694131 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc6e25c-398e-436b-8426-3360255ab479 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:32.705576 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc6e25c-398e-436b-8426-3360255ab479 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:32.705576 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc6e25c-398e-436b-8426-3360255ab479 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:32.705576 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 436/880] 10.222.0.23 () {56 vars in 1122 bytes} [Sat Jun 18 04:59:32 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 18 04:59:32.766003 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003cbf17-d736-4aa5-b968-8fc3d7534dc3 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:32.806193 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034be088-4abc-4a1c-8df0-0d9b77f66386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:32.806878 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034be088-4abc-4a1c-8df0-0d9b77f66386 None admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:32.807162 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034be088-4abc-4a1c-8df0-0d9b77f66386 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:32.807436 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034be088-4abc-4a1c-8df0-0d9b77f66386 None admin] PATH_INFO: `/v3/services` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:32.808260 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034be088-4abc-4a1c-8df0-0d9b77f66386 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:32.809685 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034be088-4abc-4a1c-8df0-0d9b77f66386 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:32.882054 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 445/881] 10.222.0.23 () {60 vars in 1141 bytes} [Sat Jun 18 04:59:32 2022] POST /identity/v3/services => generated 269 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 18 04:59:33.352322 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465787db-19a6-40a4-a53d-b6efb5818f85 None None] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:33.353018 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465787db-19a6-40a4-a53d-b6efb5818f85 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:33.353591 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465787db-19a6-40a4-a53d-b6efb5818f85 None None] PATH_INFO: `/` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:33.355427 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 437/882] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 04:59:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 04:59:33.360830 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83303f1c-5c1f-4f71-a631-4abd061d0970 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:33.361360 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83303f1c-5c1f-4f71-a631-4abd061d0970 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:33.361873 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83303f1c-5c1f-4f71-a631-4abd061d0970 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:33.391078 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-83303f1c-5c1f-4f71-a631-4abd061d0970 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 04:59:33.392533 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-83303f1c-5c1f-4f71-a631-4abd061d0970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 04:59:33.445112 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:33.467410 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 446/883] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 04:59:33 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 18 04:59:33.495443 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465787db-19a6-40a4-a53d-b6efb5818f85 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:33.544248 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:33.601848 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465787db-19a6-40a4-a53d-b6efb5818f85 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:33.637676 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b642a0-3d60-4111-bc2a-0aa7eaa34d5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:33.638411 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b642a0-3d60-4111-bc2a-0aa7eaa34d5a None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:33.638694 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b642a0-3d60-4111-bc2a-0aa7eaa34d5a None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:33.638972 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b642a0-3d60-4111-bc2a-0aa7eaa34d5a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:33.649867 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b642a0-3d60-4111-bc2a-0aa7eaa34d5a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 18 04:59:33.657887 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application [None req-20b642a0-3d60-4111-bc2a-0aa7eaa34d5a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 18 04:59:33.657887 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:33.657887 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:59:33.657887 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:59:33.657887 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:59:33.657887 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:59:33.657887 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:59:33.657887 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:59:33.657887 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:59:33.657887 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:59:33.657887 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:59:33.657887 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:59:33.657887 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 18 04:59:33.657887 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 18 04:59:33.657887 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 18 04:59:33.657887 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 18 04:59:33.658818 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 18 04:59:33.658818 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 18 04:59:33.658818 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 18 04:59:33.658818 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 18 04:59:33.658818 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:59:33.658818 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:59:33.658818 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:59:33.658818 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:59:33.658818 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:59:33.658818 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:59:33.658818 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:59:33.658818 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application with Lock( Jun 18 04:59:33.658818 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:59:33.658818 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:59:33.658818 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:59:33.658818 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:59:33.659689 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:59:33.659689 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:59:33.659689 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:59:33.659689 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:59:33.659689 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 18 04:59:33.659689 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 18 04:59:33.659689 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 18 04:59:33.659689 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 18 04:59:33.659689 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 18 04:59:33.659689 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 18 04:59:33.659689 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 18 04:59:33.659689 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 04:59:33.659689 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 438/884] 10.222.0.23 () {56 vars in 1120 bytes} [Sat Jun 18 04:59:33 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:59:33.665571 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83303f1c-5c1f-4f71-a631-4abd061d0970 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:33.775238 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83303f1c-5c1f-4f71-a631-4abd061d0970 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:33.811494 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f0385c-9413-4c7c-9728-12d184acfd0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:33.812519 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f0385c-9413-4c7c-9728-12d184acfd0a None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:33.812997 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f0385c-9413-4c7c-9728-12d184acfd0a None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:33.813490 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f0385c-9413-4c7c-9728-12d184acfd0a None admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:33.814429 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f0385c-9413-4c7c-9728-12d184acfd0a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:33.814931 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f0385c-9413-4c7c-9728-12d184acfd0a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:33.816507 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f0385c-9413-4c7c-9728-12d184acfd0a None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:33.826207 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 447/885] 10.222.0.23 () {56 vars in 1115 bytes} [Sat Jun 18 04:59:33 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:33.831374 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b642a0-3d60-4111-bc2a-0aa7eaa34d5a None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:33.938591 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b642a0-3d60-4111-bc2a-0aa7eaa34d5a None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:33.941052 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:33.973461 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808e9041-6f36-4d7f-8f4e-4a7aa3b9615f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:33.974351 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808e9041-6f36-4d7f-8f4e-4a7aa3b9615f None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:33.974981 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808e9041-6f36-4d7f-8f4e-4a7aa3b9615f None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:33.974981 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808e9041-6f36-4d7f-8f4e-4a7aa3b9615f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:33.985847 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808e9041-6f36-4d7f-8f4e-4a7aa3b9615f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:33.987290 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808e9041-6f36-4d7f-8f4e-4a7aa3b9615f None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:34.005772 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application [None req-808e9041-6f36-4d7f-8f4e-4a7aa3b9615f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 04:59:34.005772 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:34.005772 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:59:34.005772 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:59:34.005772 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:59:34.005772 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:59:34.005772 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:59:34.005772 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:59:34.005772 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:59:34.005772 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:59:34.005772 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:59:34.005772 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:59:34.005772 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 18 04:59:34.005772 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 18 04:59:34.005772 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 18 04:59:34.007104 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 18 04:59:34.007104 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:59:34.007104 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:59:34.007104 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:59:34.007104 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:59:34.007104 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:59:34.007104 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:59:34.007104 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:59:34.007104 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application with Lock( Jun 18 04:59:34.007104 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:59:34.007104 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:59:34.007104 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:59:34.007104 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:59:34.007104 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:59:34.007104 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:59:34.007104 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:59:34.007104 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:59:34.009145 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 18 04:59:34.009145 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 18 04:59:34.009145 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 04:59:34.009145 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 04:59:34.009145 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 439/886] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Jun 18 04:59:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:59:34.012518 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f0385c-9413-4c7c-9728-12d184acfd0a None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:34.061217 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:34.126589 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f0385c-9413-4c7c-9728-12d184acfd0a None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:34.163009 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434f84f8-7765-4bb3-936a-e9b80476a5fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:34.163669 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434f84f8-7765-4bb3-936a-e9b80476a5fc None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:34.163951 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434f84f8-7765-4bb3-936a-e9b80476a5fc None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:34.164222 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434f84f8-7765-4bb3-936a-e9b80476a5fc None admin] PATH_INFO: `/v3/domains` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:34.164936 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434f84f8-7765-4bb3-936a-e9b80476a5fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:34.165240 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434f84f8-7765-4bb3-936a-e9b80476a5fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:34.166633 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434f84f8-7765-4bb3-936a-e9b80476a5fc None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:34.179566 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 448/887] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Jun 18 04:59:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:34.186709 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808e9041-6f36-4d7f-8f4e-4a7aa3b9615f None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:34.199234 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e02d18f-e0de-49cf-a868-d779f5f56742 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:34.199487 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e02d18f-e0de-49cf-a868-d779f5f56742 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:34.199713 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e02d18f-e0de-49cf-a868-d779f5f56742 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:34.201270 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 449/888] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 04:59:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 18 04:59:34.206030 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bda0bf-4e14-4912-8528-342153989c68 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:34.206293 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bda0bf-4e14-4912-8528-342153989c68 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:34.206590 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bda0bf-4e14-4912-8528-342153989c68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:34.223382 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-b7bda0bf-4e14-4912-8528-342153989c68 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 04:59:34.225618 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-b7bda0bf-4e14-4912-8528-342153989c68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 04:59:34.305862 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 450/889] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 04:59:34 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 18 04:59:34.310917 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808e9041-6f36-4d7f-8f4e-4a7aa3b9615f None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:34.323957 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bda0bf-4e14-4912-8528-342153989c68 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:34.357658 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a967b89d-0b88-4313-8e1e-8dd6e5acc6ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:34.359122 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a967b89d-0b88-4313-8e1e-8dd6e5acc6ee None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:34.359401 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a967b89d-0b88-4313-8e1e-8dd6e5acc6ee None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:34.359671 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a967b89d-0b88-4313-8e1e-8dd6e5acc6ee None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:34.369469 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:34.370810 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a967b89d-0b88-4313-8e1e-8dd6e5acc6ee None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:34.373208 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a967b89d-0b88-4313-8e1e-8dd6e5acc6ee None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:34.393467 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application [None req-a967b89d-0b88-4313-8e1e-8dd6e5acc6ee None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 18 04:59:34.393467 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:34.393467 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:59:34.393467 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:59:34.393467 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:59:34.393467 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:59:34.393467 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:59:34.393467 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:59:34.393467 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:59:34.393467 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:59:34.393467 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:59:34.393467 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:59:34.393467 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 18 04:59:34.393467 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 18 04:59:34.393467 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 18 04:59:34.393467 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 18 04:59:34.394590 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:59:34.394590 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:59:34.394590 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 18 04:59:34.394590 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 18 04:59:34.394590 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 18 04:59:34.394590 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 18 04:59:34.394590 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:59:34.394590 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:59:34.394590 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:59:34.394590 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:59:34.394590 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:59:34.394590 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application with Lock( Jun 18 04:59:34.394590 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:59:34.394590 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:59:34.394590 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:59:34.394590 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:59:34.394590 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:59:34.394590 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:59:34.395547 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:59:34.395547 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:59:34.395547 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 18 04:59:34.395547 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 18 04:59:34.395547 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 18 04:59:34.395547 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 18 04:59:34.395547 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 18 04:59:34.395547 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 18 04:59:34.395547 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 18 04:59:34.395547 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 04:59:34.396731 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 440/890] 10.222.0.23 () {56 vars in 1117 bytes} [Sat Jun 18 04:59:34 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:59:34.401559 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a967b89d-0b88-4313-8e1e-8dd6e5acc6ee None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:34.427046 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:34.452907 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bda0bf-4e14-4912-8528-342153989c68 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:34.492224 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedf419e-dbe3-4c97-b064-bb3d2bf47be3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:34.493340 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedf419e-dbe3-4c97-b064-bb3d2bf47be3 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:34.493824 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedf419e-dbe3-4c97-b064-bb3d2bf47be3 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:34.494269 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedf419e-dbe3-4c97-b064-bb3d2bf47be3 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:34.495133 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedf419e-dbe3-4c97-b064-bb3d2bf47be3 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:34.496619 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedf419e-dbe3-4c97-b064-bb3d2bf47be3 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:34.510478 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-fedf419e-dbe3-4c97-b064-bb3d2bf47be3 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 18 04:59:34.510478 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:34.510478 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 18 04:59:34.510478 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 18 04:59:34.510478 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 18 04:59:34.510478 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 18 04:59:34.510478 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 18 04:59:34.510478 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 18 04:59:34.510478 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 18 04:59:34.510478 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 04:59:34.510478 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 18 04:59:34.510478 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 04:59:34.510478 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:34.510478 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:59:34.510478 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:59:34.510478 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:59:34.510478 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:59:34.511502 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:59:34.511502 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:59:34.511502 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:59:34.511502 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:59:34.511502 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:59:34.511502 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:59:34.511502 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 18 04:59:34.511502 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 18 04:59:34.511502 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 18 04:59:34.511502 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 18 04:59:34.511502 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:59:34.511502 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:59:34.511502 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:59:34.511502 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:59:34.511502 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:59:34.511502 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:59:34.512354 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:59:34.512354 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jun 18 04:59:34.512354 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:59:34.512354 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:59:34.512354 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:59:34.512354 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:59:34.512354 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:59:34.512354 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:59:34.512354 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:59:34.512354 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:59:34.512354 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 18 04:59:34.512354 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 18 04:59:34.512354 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 18 04:59:34.512354 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 04:59:34.516853 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 451/891] 10.222.0.23 () {56 vars in 1114 bytes} [Sat Jun 18 04:59:34 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 18 04:59:34.517743 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedf419e-dbe3-4c97-b064-bb3d2bf47be3 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:34.534851 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a967b89d-0b88-4313-8e1e-8dd6e5acc6ee None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:34.583486 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4086327d-ab02-4ff3-b8a2-3af33f48ef7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:34.584597 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4086327d-ab02-4ff3-b8a2-3af33f48ef7e None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:34.585499 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4086327d-ab02-4ff3-b8a2-3af33f48ef7e None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:34.585931 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4086327d-ab02-4ff3-b8a2-3af33f48ef7e None admin] PATH_INFO: `/v3/users` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:34.587175 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4086327d-ab02-4ff3-b8a2-3af33f48ef7e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:34.587883 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4086327d-ab02-4ff3-b8a2-3af33f48ef7e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:34.590186 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4086327d-ab02-4ff3-b8a2-3af33f48ef7e None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:34.620295 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 441/892] 10.222.0.23 () {56 vars in 1149 bytes} [Sat Jun 18 04:59:34 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:34.621829 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4086327d-ab02-4ff3-b8a2-3af33f48ef7e None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:34.629364 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:34.634320 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedf419e-dbe3-4c97-b064-bb3d2bf47be3 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:34.671493 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b67268-ebec-494a-b5ca-ff1076725dfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:34.672230 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b67268-ebec-494a-b5ca-ff1076725dfa None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:34.672643 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b67268-ebec-494a-b5ca-ff1076725dfa None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:34.672989 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b67268-ebec-494a-b5ca-ff1076725dfa None admin] PATH_INFO: `/v3/services` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:34.673795 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b67268-ebec-494a-b5ca-ff1076725dfa None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:34.674168 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b67268-ebec-494a-b5ca-ff1076725dfa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:34.675715 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b67268-ebec-494a-b5ca-ff1076725dfa None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:34.685633 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 452/893] 10.222.0.23 () {56 vars in 1114 bytes} [Sat Jun 18 04:59:34 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:34.688712 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b67268-ebec-494a-b5ca-ff1076725dfa None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:34.728390 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:34.762952 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4086327d-ab02-4ff3-b8a2-3af33f48ef7e None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:34.807914 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b67268-ebec-494a-b5ca-ff1076725dfa None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:34.812868 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53782889-5fb2-46a5-b18a-31d49552f86d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:34.813723 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53782889-5fb2-46a5-b18a-31d49552f86d None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:34.814115 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53782889-5fb2-46a5-b18a-31d49552f86d None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:34.814492 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53782889-5fb2-46a5-b18a-31d49552f86d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:34.844570 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b64141d-c751-42bf-9191-a4b7247769ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:34.845520 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b64141d-c751-42bf-9191-a4b7247769ff None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:34.845944 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b64141d-c751-42bf-9191-a4b7247769ff None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:34.846415 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b64141d-c751-42bf-9191-a4b7247769ff None admin] PATH_INFO: `/v3/services` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:34.847266 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b64141d-c751-42bf-9191-a4b7247769ff None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:34.847757 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b64141d-c751-42bf-9191-a4b7247769ff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:34.849356 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b64141d-c751-42bf-9191-a4b7247769ff None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:34.859570 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 453/894] 10.222.0.23 () {56 vars in 1114 bytes} [Sat Jun 18 04:59:34 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 18 04:59:34.860939 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b64141d-c751-42bf-9191-a4b7247769ff None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:34.889489 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:34.898446 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53782889-5fb2-46a5-b18a-31d49552f86d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:34.905218 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53782889-5fb2-46a5-b18a-31d49552f86d None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:34.970963 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b64141d-c751-42bf-9191-a4b7247769ff None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:34.979311 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:35.007017 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a7f30f-79e6-44e2-924a-68fc31209eb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:35.007676 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a7f30f-79e6-44e2-924a-68fc31209eb9 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:35.007960 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a7f30f-79e6-44e2-924a-68fc31209eb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:35.008233 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a7f30f-79e6-44e2-924a-68fc31209eb9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:35.009020 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a7f30f-79e6-44e2-924a-68fc31209eb9 None admin] RBAC: Adding query filter params (interface=public, service_id=0bed8286c0a6458fa23299b379d6e3d5, region_id=RegionOne) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:35.009341 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a7f30f-79e6-44e2-924a-68fc31209eb9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:35.010753 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a7f30f-79e6-44e2-924a-68fc31209eb9 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:35.020295 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 454/895] 10.222.0.23 () {56 vars in 1251 bytes} [Sat Jun 18 04:59:34 2022] GET /identity/v3/endpoints?service_id=0bed8286c0a6458fa23299b379d6e3d5&interface=public®ion_id=RegionOne => generated 194 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:35.020295 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13238d26-7287-4ca6-88dc-8883de714637 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:35.020295 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13238d26-7287-4ca6-88dc-8883de714637 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:35.020295 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13238d26-7287-4ca6-88dc-8883de714637 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:35.020619 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 442/896] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Jun 18 04:59:34 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 18 04:59:35.021667 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53782889-5fb2-46a5-b18a-31d49552f86d None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:35.023896 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 455/897] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 04:59:35 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 04:59:35.024595 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13238d26-7287-4ca6-88dc-8883de714637 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:35.120149 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:35.131022 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53782889-5fb2-46a5-b18a-31d49552f86d None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:35.131275 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13238d26-7287-4ca6-88dc-8883de714637 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:35.165715 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01785d9-e084-433c-b22a-8e6a45000c6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:35.165904 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a3b790-c633-444e-93bd-4c12d8e1c283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:35.166497 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01785d9-e084-433c-b22a-8e6a45000c6c None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:35.166597 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a3b790-c633-444e-93bd-4c12d8e1c283 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:35.166781 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01785d9-e084-433c-b22a-8e6a45000c6c None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:35.166944 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a3b790-c633-444e-93bd-4c12d8e1c283 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:35.167055 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01785d9-e084-433c-b22a-8e6a45000c6c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:35.167131 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a3b790-c633-444e-93bd-4c12d8e1c283 None admin] PATH_INFO: `/v3/services` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:35.167712 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a3b790-c633-444e-93bd-4c12d8e1c283 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:35.169073 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a3b790-c633-444e-93bd-4c12d8e1c283 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:35.174661 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 456/898] 10.222.0.23 () {56 vars in 1087 bytes} [Sat Jun 18 04:59:35 2022] GET /identity/v3/services => generated 1268 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 04:59:35.175804 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2694e343-1c4c-45e9-8e73-d1cc984a3546 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:35.176090 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2694e343-1c4c-45e9-8e73-d1cc984a3546 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:35.176363 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2694e343-1c4c-45e9-8e73-d1cc984a3546 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:35.178132 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01785d9-e084-433c-b22a-8e6a45000c6c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:35.179867 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01785d9-e084-433c-b22a-8e6a45000c6c None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:35.194345 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-2694e343-1c4c-45e9-8e73-d1cc984a3546 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 04:59:35.195571 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-2694e343-1c4c-45e9-8e73-d1cc984a3546 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 04:59:35.201205 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application [None req-c01785d9-e084-433c-b22a-8e6a45000c6c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 04:59:35.201205 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:35.201205 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:59:35.201205 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:59:35.201205 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:59:35.201205 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:59:35.201205 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:59:35.201205 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:59:35.201205 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:59:35.201205 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:59:35.201205 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:59:35.201205 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:59:35.201205 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 18 04:59:35.201205 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 18 04:59:35.201205 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 18 04:59:35.202292 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 18 04:59:35.202292 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:59:35.202292 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:59:35.202292 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:59:35.202292 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:59:35.202292 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:59:35.202292 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:59:35.202292 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:59:35.202292 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application with Lock( Jun 18 04:59:35.202292 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:59:35.202292 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:59:35.202292 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:59:35.202292 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:59:35.202292 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:59:35.202292 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:59:35.202292 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:59:35.202292 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:59:35.203156 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 18 04:59:35.203156 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 18 04:59:35.203156 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 04:59:35.203156 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 04:59:35.203820 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 443/899] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Jun 18 04:59:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:59:35.205546 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab228784-404f-4b3c-affa-366a27e1d00f None None] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:35.205820 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab228784-404f-4b3c-affa-366a27e1d00f None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:35.206466 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab228784-404f-4b3c-affa-366a27e1d00f None None] PATH_INFO: `/` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:35.208094 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 444/900] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 04:59:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 04:59:35.209266 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab228784-404f-4b3c-affa-366a27e1d00f None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:35.242559 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:35.272876 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 457/901] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 04:59:35 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 18 04:59:35.274297 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a578a06-89cf-4f49-a2f4-e62b5a5c20e5 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:35.274592 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a578a06-89cf-4f49-a2f4-e62b5a5c20e5 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:35.274868 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a578a06-89cf-4f49-a2f4-e62b5a5c20e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:35.294672 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-3a578a06-89cf-4f49-a2f4-e62b5a5c20e5 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 04:59:35.296406 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-3a578a06-89cf-4f49-a2f4-e62b5a5c20e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 04:59:35.348449 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab228784-404f-4b3c-affa-366a27e1d00f None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:35.373755 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 458/902] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 04:59:35 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 18 04:59:35.374407 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a578a06-89cf-4f49-a2f4-e62b5a5c20e5 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:35.394526 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:35.410657 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea14019-82d1-42fe-ae3e-712dd4a4d789 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:35.411315 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea14019-82d1-42fe-ae3e-712dd4a4d789 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:35.411622 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea14019-82d1-42fe-ae3e-712dd4a4d789 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:35.412479 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea14019-82d1-42fe-ae3e-712dd4a4d789 None admin] PATH_INFO: `/v3/domains` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:35.413148 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea14019-82d1-42fe-ae3e-712dd4a4d789 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:35.413497 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea14019-82d1-42fe-ae3e-712dd4a4d789 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:35.414900 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea14019-82d1-42fe-ae3e-712dd4a4d789 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:35.428840 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 445/903] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Jun 18 04:59:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:35.429966 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea14019-82d1-42fe-ae3e-712dd4a4d789 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:35.497569 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a578a06-89cf-4f49-a2f4-e62b5a5c20e5 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:35.520413 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:35.536203 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea14019-82d1-42fe-ae3e-712dd4a4d789 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:35.537204 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78eb342a-50d9-4b58-b9e5-ad9c3b739dd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:35.538977 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78eb342a-50d9-4b58-b9e5-ad9c3b739dd5 None admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:35.538977 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78eb342a-50d9-4b58-b9e5-ad9c3b739dd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:35.538977 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78eb342a-50d9-4b58-b9e5-ad9c3b739dd5 None admin] PATH_INFO: `/v3/services` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:35.539317 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78eb342a-50d9-4b58-b9e5-ad9c3b739dd5 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:35.540649 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78eb342a-50d9-4b58-b9e5-ad9c3b739dd5 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:35.573201 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e76812f-d3f3-49a5-88a8-74d42dd3a785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:35.574029 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e76812f-d3f3-49a5-88a8-74d42dd3a785 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:35.574311 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e76812f-d3f3-49a5-88a8-74d42dd3a785 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:35.574583 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e76812f-d3f3-49a5-88a8-74d42dd3a785 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:35.575232 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e76812f-d3f3-49a5-88a8-74d42dd3a785 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:35.576619 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e76812f-d3f3-49a5-88a8-74d42dd3a785 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:35.588730 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application [None req-3e76812f-d3f3-49a5-88a8-74d42dd3a785 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 18 04:59:35.588730 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:35.588730 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 18 04:59:35.588730 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 18 04:59:35.588730 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 18 04:59:35.588730 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 18 04:59:35.588730 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 18 04:59:35.588730 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 18 04:59:35.588730 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 18 04:59:35.588730 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 04:59:35.588730 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 18 04:59:35.588730 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 04:59:35.588730 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:35.588730 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:59:35.588730 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:59:35.588730 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:59:35.589751 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:59:35.589751 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:59:35.589751 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:59:35.589751 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:59:35.589751 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:59:35.589751 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:59:35.589751 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:59:35.589751 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 18 04:59:35.589751 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 18 04:59:35.589751 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 18 04:59:35.589751 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 18 04:59:35.589751 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:59:35.589751 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:59:35.589751 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:59:35.589751 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:59:35.589751 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:59:35.590621 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:59:35.590621 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:59:35.590621 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application with Lock( Jun 18 04:59:35.590621 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:59:35.590621 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:59:35.590621 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:59:35.590621 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:59:35.590621 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:59:35.590621 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:59:35.590621 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:59:35.590621 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:59:35.590621 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 18 04:59:35.590621 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 18 04:59:35.590621 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 18 04:59:35.590621 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 04:59:35.590621 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 446/904] 10.222.0.23 () {56 vars in 1132 bytes} [Sat Jun 18 04:59:35 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:59:35.591459 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e76812f-d3f3-49a5-88a8-74d42dd3a785 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:35.633423 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 459/905] 10.222.0.23 () {60 vars in 1141 bytes} [Sat Jun 18 04:59:35 2022] POST /identity/v3/services => generated 241 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 04:59:35.633423 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78eb342a-50d9-4b58-b9e5-ad9c3b739dd5 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:35.721116 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e76812f-d3f3-49a5-88a8-74d42dd3a785 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:35.728870 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:35.763575 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78eb342a-50d9-4b58-b9e5-ad9c3b739dd5 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:35.769526 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdb6993-2b19-4763-94da-582953a69f36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:35.771047 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdb6993-2b19-4763-94da-582953a69f36 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:35.771492 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdb6993-2b19-4763-94da-582953a69f36 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:35.772104 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdb6993-2b19-4763-94da-582953a69f36 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:35.780428 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdb6993-2b19-4763-94da-582953a69f36 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:35.782285 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdb6993-2b19-4763-94da-582953a69f36 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:35.800561 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application [None req-5cdb6993-2b19-4763-94da-582953a69f36 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 18 04:59:35.800561 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:35.800561 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:59:35.800561 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:59:35.800561 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:59:35.800561 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:59:35.800561 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:59:35.800561 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:59:35.800561 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:59:35.800561 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:59:35.800561 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:59:35.800561 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:59:35.800561 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 18 04:59:35.800561 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 18 04:59:35.800561 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 18 04:59:35.801742 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 18 04:59:35.801742 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:59:35.801742 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:59:35.801742 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:59:35.801742 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:59:35.801742 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:59:35.801742 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:59:35.801742 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:59:35.801742 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application with Lock( Jun 18 04:59:35.801742 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:59:35.801742 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:59:35.801742 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:59:35.801742 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:59:35.801742 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:59:35.801742 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:59:35.801742 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:59:35.801742 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:59:35.803479 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 18 04:59:35.803479 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 18 04:59:35.803479 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 18 04:59:35.803479 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 18 04:59:35.803479 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 18 04:59:35.803479 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 18 04:59:35.803479 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 18 04:59:35.803479 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 04:59:35.805630 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d385215c-9992-40b5-9374-74efdf4a45c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:35.805810 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 447/906] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Jun 18 04:59:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:59:35.806214 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d385215c-9992-40b5-9374-74efdf4a45c4 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:35.806495 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d385215c-9992-40b5-9374-74efdf4a45c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:35.806770 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d385215c-9992-40b5-9374-74efdf4a45c4 None admin] PATH_INFO: `/v3/services` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:35.807478 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d385215c-9992-40b5-9374-74efdf4a45c4 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:35.807783 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d385215c-9992-40b5-9374-74efdf4a45c4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:35.809150 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d385215c-9992-40b5-9374-74efdf4a45c4 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:35.814751 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 460/907] 10.222.0.23 () {56 vars in 1126 bytes} [Sat Jun 18 04:59:35 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:35.815089 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdb6993-2b19-4763-94da-582953a69f36 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:35.821068 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d385215c-9992-40b5-9374-74efdf4a45c4 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:35.825338 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:35.932744 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d385215c-9992-40b5-9374-74efdf4a45c4 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:35.948865 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdb6993-2b19-4763-94da-582953a69f36 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:35.972744 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d483e36-5063-4189-9b97-d95441a62eb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:35.973560 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d483e36-5063-4189-9b97-d95441a62eb7 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:35.973966 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d483e36-5063-4189-9b97-d95441a62eb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:35.975176 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d483e36-5063-4189-9b97-d95441a62eb7 None admin] PATH_INFO: `/v3/services` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:35.975960 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d483e36-5063-4189-9b97-d95441a62eb7 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:35.976339 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d483e36-5063-4189-9b97-d95441a62eb7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:35.977860 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d483e36-5063-4189-9b97-d95441a62eb7 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:35.979480 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:35.984213 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 461/908] 10.222.0.23 () {56 vars in 1126 bytes} [Sat Jun 18 04:59:35 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:35.990296 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d483e36-5063-4189-9b97-d95441a62eb7 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:35.996447 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8c810b-ebb7-44a5-98d9-099fe09a4b7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:35.997359 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8c810b-ebb7-44a5-98d9-099fe09a4b7d None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:35.997834 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8c810b-ebb7-44a5-98d9-099fe09a4b7d None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:35.998284 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8c810b-ebb7-44a5-98d9-099fe09a4b7d None admin] PATH_INFO: `/v3/projects` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:35.999236 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8c810b-ebb7-44a5-98d9-099fe09a4b7d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:36.000183 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8c810b-ebb7-44a5-98d9-099fe09a4b7d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:36.002232 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8c810b-ebb7-44a5-98d9-099fe09a4b7d None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:36.019179 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 448/909] 10.222.0.23 () {56 vars in 1148 bytes} [Sat Jun 18 04:59:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 18 04:59:36.029489 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8c810b-ebb7-44a5-98d9-099fe09a4b7d None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:36.106882 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d483e36-5063-4189-9b97-d95441a62eb7 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:36.118033 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:36.152255 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495aa74f-48a8-4f14-bdf4-39977809a48a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:36.154531 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495aa74f-48a8-4f14-bdf4-39977809a48a None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:36.155265 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495aa74f-48a8-4f14-bdf4-39977809a48a None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:36.155795 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495aa74f-48a8-4f14-bdf4-39977809a48a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:36.156777 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495aa74f-48a8-4f14-bdf4-39977809a48a None admin] RBAC: Adding query filter params (interface=public, service_id=f8a7cf718101440b9ba6c6eef3b85486, region_id=RegionOne) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:36.157214 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8c810b-ebb7-44a5-98d9-099fe09a4b7d None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:36.157491 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495aa74f-48a8-4f14-bdf4-39977809a48a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:36.159424 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495aa74f-48a8-4f14-bdf4-39977809a48a None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:36.167560 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 462/910] 10.222.0.23 () {56 vars in 1251 bytes} [Sat Jun 18 04:59:35 2022] GET /identity/v3/endpoints?service_id=f8a7cf718101440b9ba6c6eef3b85486&interface=public®ion_id=RegionOne => generated 194 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 18 04:59:36.174001 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495aa74f-48a8-4f14-bdf4-39977809a48a None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:36.196130 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e34e820-d655-471d-b733-c02f15718dff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:36.196962 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e34e820-d655-471d-b733-c02f15718dff None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:36.197086 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e34e820-d655-471d-b733-c02f15718dff None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:36.197322 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e34e820-d655-471d-b733-c02f15718dff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:36.198192 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e34e820-d655-471d-b733-c02f15718dff None admin] RBAC: Adding query filter params (role.id=2495da35026945aabd29678096996e70, scope.project.id=c95f8f3084c5472cb8364b7c57207394, user.id=e817fafd0c1b4bf9944481cf116f1260) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:36.198561 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e34e820-d655-471d-b733-c02f15718dff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:36.199995 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e34e820-d655-471d-b733-c02f15718dff None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:36.213688 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 449/911] 10.222.0.23 () {56 vars in 1374 bytes} [Sat Jun 18 04:59:36 2022] GET /identity/v3/role_assignments?user.id=e817fafd0c1b4bf9944481cf116f1260&scope.project.id=c95f8f3084c5472cb8364b7c57207394&role.id=2495da35026945aabd29678096996e70 => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 18 04:59:36.253671 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:36.300366 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495aa74f-48a8-4f14-bdf4-39977809a48a None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:36.341288 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed216f9c-bb2c-4d7b-93b3-97878ac9772d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:36.342438 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed216f9c-bb2c-4d7b-93b3-97878ac9772d None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:36.342932 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed216f9c-bb2c-4d7b-93b3-97878ac9772d None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:36.343417 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed216f9c-bb2c-4d7b-93b3-97878ac9772d None admin] PATH_INFO: `/v3/services` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:36.344368 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed216f9c-bb2c-4d7b-93b3-97878ac9772d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:36.346035 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed216f9c-bb2c-4d7b-93b3-97878ac9772d None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:36.353714 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 463/912] 10.222.0.23 () {56 vars in 1087 bytes} [Sat Jun 18 04:59:36 2022] GET /identity/v3/services => generated 1497 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 18 04:59:37.293011 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553bfb8d-f1fb-42f7-9d3c-3f30e4e6376d None None] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:37.293661 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553bfb8d-f1fb-42f7-9d3c-3f30e4e6376d None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:37.293661 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553bfb8d-f1fb-42f7-9d3c-3f30e4e6376d None None] PATH_INFO: `/` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:37.295261 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 450/913] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 04:59:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 04:59:37.300476 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772cd819-271b-466f-a30b-331159e9ec5c None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:37.300895 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772cd819-271b-466f-a30b-331159e9ec5c None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:37.301068 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772cd819-271b-466f-a30b-331159e9ec5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:37.319037 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-772cd819-271b-466f-a30b-331159e9ec5c None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 04:59:37.320239 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-772cd819-271b-466f-a30b-331159e9ec5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 04:59:37.397490 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 464/914] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 04:59:37 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 18 04:59:37.417767 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553bfb8d-f1fb-42f7-9d3c-3f30e4e6376d None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:37.525733 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553bfb8d-f1fb-42f7-9d3c-3f30e4e6376d None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:37.557233 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:37.563399 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c834ef98-5c4b-4149-a0d2-14ad9239df45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:37.564368 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c834ef98-5c4b-4149-a0d2-14ad9239df45 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:37.564859 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c834ef98-5c4b-4149-a0d2-14ad9239df45 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:37.565295 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c834ef98-5c4b-4149-a0d2-14ad9239df45 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:37.566115 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c834ef98-5c4b-4149-a0d2-14ad9239df45 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:37.567630 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c834ef98-5c4b-4149-a0d2-14ad9239df45 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:37.579626 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application [None req-c834ef98-5c4b-4149-a0d2-14ad9239df45 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 18 04:59:37.579626 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:37.579626 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 18 04:59:37.579626 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 18 04:59:37.579626 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 18 04:59:37.579626 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 18 04:59:37.579626 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 18 04:59:37.579626 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 18 04:59:37.579626 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 18 04:59:37.579626 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 04:59:37.579626 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 18 04:59:37.579626 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 04:59:37.579626 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:37.579626 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:59:37.579626 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:59:37.579626 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:59:37.579626 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:59:37.582277 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:59:37.582277 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:59:37.582277 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:59:37.582277 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:59:37.582277 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:59:37.582277 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:59:37.582277 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 18 04:59:37.582277 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 18 04:59:37.582277 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 18 04:59:37.582277 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 18 04:59:37.582277 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:59:37.582277 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:59:37.582277 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:59:37.582277 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:59:37.582277 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:59:37.582277 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:59:37.583175 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:59:37.583175 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application with Lock( Jun 18 04:59:37.583175 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:59:37.583175 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:59:37.583175 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:59:37.583175 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:59:37.583175 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:59:37.583175 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:59:37.583175 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:59:37.583175 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:59:37.583175 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 18 04:59:37.583175 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 18 04:59:37.583175 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 18 04:59:37.583175 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 04:59:37.583175 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 451/915] 10.222.0.23 () {56 vars in 1114 bytes} [Sat Jun 18 04:59:37 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:59:37.587752 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772cd819-271b-466f-a30b-331159e9ec5c None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:37.647479 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:37.692803 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772cd819-271b-466f-a30b-331159e9ec5c None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:37.729547 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d46223-b7f2-4f56-8eb2-362dc2fdb03d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:37.730501 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d46223-b7f2-4f56-8eb2-362dc2fdb03d None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:37.730666 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d46223-b7f2-4f56-8eb2-362dc2fdb03d None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:37.730970 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d46223-b7f2-4f56-8eb2-362dc2fdb03d None admin] PATH_INFO: `/v3/services` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:37.731645 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d46223-b7f2-4f56-8eb2-362dc2fdb03d None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:37.732114 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d46223-b7f2-4f56-8eb2-362dc2fdb03d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:37.733331 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d46223-b7f2-4f56-8eb2-362dc2fdb03d None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:37.738719 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 465/916] 10.222.0.23 () {56 vars in 1114 bytes} [Sat Jun 18 04:59:37 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 18 04:59:37.746555 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c834ef98-5c4b-4149-a0d2-14ad9239df45 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:37.776749 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de8d65e-92e3-49e9-a809-ecec4fa90e95 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:37.777082 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de8d65e-92e3-49e9-a809-ecec4fa90e95 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:37.777322 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de8d65e-92e3-49e9-a809-ecec4fa90e95 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:37.779710 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 466/917] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 04:59:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 18 04:59:37.786431 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6d0a3b-4161-487b-9c29-05274fbf7360 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:37.787153 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6d0a3b-4161-487b-9c29-05274fbf7360 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:37.787655 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6d0a3b-4161-487b-9c29-05274fbf7360 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:37.805971 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-dd6d0a3b-4161-487b-9c29-05274fbf7360 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 04:59:37.807540 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-dd6d0a3b-4161-487b-9c29-05274fbf7360 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 04:59:37.872740 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c834ef98-5c4b-4149-a0d2-14ad9239df45 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:37.885481 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 467/918] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 04:59:37 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 18 04:59:37.903743 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6d0a3b-4161-487b-9c29-05274fbf7360 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:37.912434 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5f5328-3ba3-4686-b81e-0911ac949aed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:37.913157 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5f5328-3ba3-4686-b81e-0911ac949aed None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:37.913464 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5f5328-3ba3-4686-b81e-0911ac949aed None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:37.913747 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5f5328-3ba3-4686-b81e-0911ac949aed None admin] PATH_INFO: `/v3/services` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:37.914880 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5f5328-3ba3-4686-b81e-0911ac949aed None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:37.915217 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5f5328-3ba3-4686-b81e-0911ac949aed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:37.916587 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5f5328-3ba3-4686-b81e-0911ac949aed None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:37.923747 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 452/919] 10.222.0.23 () {56 vars in 1114 bytes} [Sat Jun 18 04:59:37 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:37.929086 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5f5328-3ba3-4686-b81e-0911ac949aed None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:37.962367 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:38.002881 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:38.030833 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6d0a3b-4161-487b-9c29-05274fbf7360 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:38.066166 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5f5328-3ba3-4686-b81e-0911ac949aed None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:38.072454 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497de8f4-bb37-441e-a5c1-ec122118bd9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:38.073411 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497de8f4-bb37-441e-a5c1-ec122118bd9a None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:38.073799 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497de8f4-bb37-441e-a5c1-ec122118bd9a None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:38.074172 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497de8f4-bb37-441e-a5c1-ec122118bd9a None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:38.074932 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497de8f4-bb37-441e-a5c1-ec122118bd9a None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:38.076405 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497de8f4-bb37-441e-a5c1-ec122118bd9a None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:38.089010 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-497de8f4-bb37-441e-a5c1-ec122118bd9a None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 18 04:59:38.089010 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:38.089010 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 18 04:59:38.089010 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 18 04:59:38.089010 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 18 04:59:38.089010 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 18 04:59:38.089010 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 18 04:59:38.089010 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 18 04:59:38.089010 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 18 04:59:38.089010 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 04:59:38.089010 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 18 04:59:38.089010 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 04:59:38.089010 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:38.089010 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:59:38.089010 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:59:38.089010 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:59:38.089010 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:59:38.090280 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:59:38.090280 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:59:38.090280 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:59:38.090280 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:59:38.090280 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:59:38.090280 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:59:38.090280 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 18 04:59:38.090280 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 18 04:59:38.090280 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 18 04:59:38.090280 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 18 04:59:38.090280 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:59:38.090280 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:59:38.090280 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:59:38.090280 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:59:38.090280 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:59:38.090280 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:59:38.091300 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:59:38.091300 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jun 18 04:59:38.091300 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:59:38.091300 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:59:38.091300 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:59:38.091300 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:59:38.091300 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:59:38.091300 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:59:38.091300 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:59:38.091300 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:59:38.091300 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 18 04:59:38.091300 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 18 04:59:38.091300 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 18 04:59:38.091300 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 04:59:38.098114 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 468/920] 10.222.0.23 () {56 vars in 1126 bytes} [Sat Jun 18 04:59:37 2022] GET /identity/v3/services/object-store => generated 93 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:59:38.100081 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497de8f4-bb37-441e-a5c1-ec122118bd9a None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:38.109988 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ed5eb3-7a9c-4084-9eff-7b0ddb690789 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:38.111305 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ed5eb3-7a9c-4084-9eff-7b0ddb690789 None admin] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:38.111757 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ed5eb3-7a9c-4084-9eff-7b0ddb690789 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:38.112179 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ed5eb3-7a9c-4084-9eff-7b0ddb690789 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:38.113443 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ed5eb3-7a9c-4084-9eff-7b0ddb690789 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:38.115046 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ed5eb3-7a9c-4084-9eff-7b0ddb690789 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:38.181256 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 453/921] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Jun 18 04:59:37 2022] POST /identity/v3/endpoints => generated 346 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 04:59:38.223243 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497de8f4-bb37-441e-a5c1-ec122118bd9a None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:38.242045 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:38.274701 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccf0175-8ca0-4e64-9fab-65efdd4b6d44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:38.275812 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccf0175-8ca0-4e64-9fab-65efdd4b6d44 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:38.276376 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccf0175-8ca0-4e64-9fab-65efdd4b6d44 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:38.276893 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccf0175-8ca0-4e64-9fab-65efdd4b6d44 None admin] PATH_INFO: `/v3/services` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:38.277863 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccf0175-8ca0-4e64-9fab-65efdd4b6d44 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:38.278623 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccf0175-8ca0-4e64-9fab-65efdd4b6d44 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:38.280249 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccf0175-8ca0-4e64-9fab-65efdd4b6d44 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:38.291610 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 469/922] 10.222.0.23 () {56 vars in 1122 bytes} [Sat Jun 18 04:59:38 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 18 04:59:38.292625 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ed5eb3-7a9c-4084-9eff-7b0ddb690789 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:38.300964 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df497f3e-5656-4731-a64b-2418eb34cec3 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:38.301538 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df497f3e-5656-4731-a64b-2418eb34cec3 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:38.301538 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df497f3e-5656-4731-a64b-2418eb34cec3 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:38.303025 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 470/923] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 04:59:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 18 04:59:38.308760 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05dbdb5-0601-49db-95dc-422b8985b05c None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:38.309308 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05dbdb5-0601-49db-95dc-422b8985b05c None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:38.309767 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05dbdb5-0601-49db-95dc-422b8985b05c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:38.327586 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-f05dbdb5-0601-49db-95dc-422b8985b05c None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 04:59:38.329064 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-f05dbdb5-0601-49db-95dc-422b8985b05c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 04:59:38.424427 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ed5eb3-7a9c-4084-9eff-7b0ddb690789 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:38.424921 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 471/924] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 04:59:38 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 18 04:59:38.439286 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:38.448186 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05dbdb5-0601-49db-95dc-422b8985b05c None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:38.461801 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193f3cda-78a5-4add-8ada-378db77688df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:38.462609 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193f3cda-78a5-4add-8ada-378db77688df None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:38.463099 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193f3cda-78a5-4add-8ada-378db77688df None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:38.463473 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193f3cda-78a5-4add-8ada-378db77688df None admin] PATH_INFO: `/v3/services` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:38.464404 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193f3cda-78a5-4add-8ada-378db77688df None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:38.464835 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193f3cda-78a5-4add-8ada-378db77688df None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:38.466422 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193f3cda-78a5-4add-8ada-378db77688df None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:38.472210 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 454/925] 10.222.0.23 () {56 vars in 1122 bytes} [Sat Jun 18 04:59:38 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:38.478295 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193f3cda-78a5-4add-8ada-378db77688df None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:38.555709 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05dbdb5-0601-49db-95dc-422b8985b05c None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:38.558823 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:38.594118 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193f3cda-78a5-4add-8ada-378db77688df None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:38.594513 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616e5109-5a87-435a-a2a9-41c0b54d0aa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:38.595351 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616e5109-5a87-435a-a2a9-41c0b54d0aa4 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:38.595632 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616e5109-5a87-435a-a2a9-41c0b54d0aa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:38.595908 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616e5109-5a87-435a-a2a9-41c0b54d0aa4 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:38.607228 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616e5109-5a87-435a-a2a9-41c0b54d0aa4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 18 04:59:38.615270 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-616e5109-5a87-435a-a2a9-41c0b54d0aa4 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 18 04:59:38.615270 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:38.615270 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:59:38.615270 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:59:38.615270 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:59:38.615270 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:59:38.615270 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:59:38.615270 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:59:38.615270 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:59:38.615270 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:59:38.615270 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:59:38.615270 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:59:38.615270 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 18 04:59:38.615270 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 18 04:59:38.615270 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 18 04:59:38.615270 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 18 04:59:38.616649 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 18 04:59:38.616649 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 18 04:59:38.616649 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 18 04:59:38.616649 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 18 04:59:38.616649 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:59:38.616649 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:59:38.616649 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:59:38.616649 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:59:38.616649 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:59:38.616649 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:59:38.616649 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:59:38.616649 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jun 18 04:59:38.616649 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:59:38.616649 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:59:38.616649 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:59:38.616649 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:59:38.617544 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:59:38.617544 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:59:38.617544 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:59:38.617544 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:59:38.617544 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 18 04:59:38.617544 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 18 04:59:38.617544 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 18 04:59:38.617544 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 18 04:59:38.617544 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 18 04:59:38.617544 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 18 04:59:38.617544 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 18 04:59:38.617544 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 04:59:38.617544 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 472/926] 10.222.0.23 () {56 vars in 1120 bytes} [Sat Jun 18 04:59:38 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 18 04:59:38.618396 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d95ec10-2989-4fe3-9b9f-6aba1aa79beb None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:38.618689 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d95ec10-2989-4fe3-9b9f-6aba1aa79beb None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:38.618964 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d95ec10-2989-4fe3-9b9f-6aba1aa79beb None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:38.624412 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 473/927] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 04:59:38 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 04:59:38.625057 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d95ec10-2989-4fe3-9b9f-6aba1aa79beb None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:38.639057 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a35f9fb-682d-4f86-aa3f-f5ac12312eee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:38.639966 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a35f9fb-682d-4f86-aa3f-f5ac12312eee None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:38.640508 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a35f9fb-682d-4f86-aa3f-f5ac12312eee None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:38.640951 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a35f9fb-682d-4f86-aa3f-f5ac12312eee None admin] PATH_INFO: `/v3/endpoints` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:38.641969 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a35f9fb-682d-4f86-aa3f-f5ac12312eee None admin] RBAC: Adding query filter params (interface=public, service_id=b4774d3f31b84eba91afda98aa15fad2, region_id=RegionOne) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:38.642461 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a35f9fb-682d-4f86-aa3f-f5ac12312eee None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:38.644097 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a35f9fb-682d-4f86-aa3f-f5ac12312eee None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:38.651239 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 455/928] 10.222.0.23 () {56 vars in 1251 bytes} [Sat Jun 18 04:59:38 2022] GET /identity/v3/endpoints?service_id=b4774d3f31b84eba91afda98aa15fad2&interface=public®ion_id=RegionOne => generated 194 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 18 04:59:38.652419 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa0f2c7-1f94-417b-9ece-ba8b2c14870b None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:38.652698 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa0f2c7-1f94-417b-9ece-ba8b2c14870b None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:38.652970 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa0f2c7-1f94-417b-9ece-ba8b2c14870b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:38.669452 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-bfa0f2c7-1f94-417b-9ece-ba8b2c14870b None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 04:59:38.670634 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-bfa0f2c7-1f94-417b-9ece-ba8b2c14870b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 04:59:38.707695 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:38.740397 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 456/929] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 04:59:38 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 18 04:59:38.741779 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa0f2c7-1f94-417b-9ece-ba8b2c14870b None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:38.750742 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d95ec10-2989-4fe3-9b9f-6aba1aa79beb None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:38.796784 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f818795b-44e6-4e15-8f15-e3b08fea6c06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:38.797811 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f818795b-44e6-4e15-8f15-e3b08fea6c06 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:38.798263 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f818795b-44e6-4e15-8f15-e3b08fea6c06 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:38.798699 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f818795b-44e6-4e15-8f15-e3b08fea6c06 None admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:38.799600 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f818795b-44e6-4e15-8f15-e3b08fea6c06 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:38.800062 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f818795b-44e6-4e15-8f15-e3b08fea6c06 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:38.801610 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f818795b-44e6-4e15-8f15-e3b08fea6c06 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:38.811471 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 474/930] 10.222.0.23 () {56 vars in 1115 bytes} [Sat Jun 18 04:59:38 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:38.812473 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f818795b-44e6-4e15-8f15-e3b08fea6c06 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:38.842359 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:38.856730 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa0f2c7-1f94-417b-9ece-ba8b2c14870b None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:38.891268 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8af403-dd02-4fc3-9d22-a486e37abe49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:38.891924 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8af403-dd02-4fc3-9d22-a486e37abe49 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:38.892203 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8af403-dd02-4fc3-9d22-a486e37abe49 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:38.892476 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8af403-dd02-4fc3-9d22-a486e37abe49 None admin] PATH_INFO: `/v3/services` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:38.893150 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8af403-dd02-4fc3-9d22-a486e37abe49 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:38.894535 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8af403-dd02-4fc3-9d22-a486e37abe49 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:38.900927 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 457/931] 10.222.0.23 () {56 vars in 1087 bytes} [Sat Jun 18 04:59:38 2022] GET /identity/v3/services => generated 1497 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 04:59:38.906488 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8af403-dd02-4fc3-9d22-a486e37abe49 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:38.922619 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f818795b-44e6-4e15-8f15-e3b08fea6c06 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:38.964781 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f76d2d-c521-4d25-8dc0-3ea52607b0d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:38.965707 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f76d2d-c521-4d25-8dc0-3ea52607b0d2 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:38.966079 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f76d2d-c521-4d25-8dc0-3ea52607b0d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:38.966367 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f76d2d-c521-4d25-8dc0-3ea52607b0d2 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:38.966733 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:38.967041 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f76d2d-c521-4d25-8dc0-3ea52607b0d2 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:38.968469 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f76d2d-c521-4d25-8dc0-3ea52607b0d2 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:38.982851 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-f3f76d2d-c521-4d25-8dc0-3ea52607b0d2 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 18 04:59:38.982851 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:38.982851 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 18 04:59:38.982851 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 18 04:59:38.982851 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 18 04:59:38.982851 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 18 04:59:38.982851 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 18 04:59:38.982851 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 18 04:59:38.982851 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 18 04:59:38.982851 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 04:59:38.982851 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 18 04:59:38.982851 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 04:59:38.982851 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:38.982851 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:59:38.982851 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:59:38.982851 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:59:38.983881 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:59:38.983881 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:59:38.983881 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:59:38.983881 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:59:38.983881 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:59:38.983881 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:59:38.983881 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:59:38.983881 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 18 04:59:38.983881 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 18 04:59:38.983881 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 18 04:59:38.983881 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 18 04:59:38.983881 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:59:38.983881 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:59:38.983881 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:59:38.983881 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:59:38.983881 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:59:38.984718 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:59:38.984718 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:59:38.984718 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jun 18 04:59:38.984718 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:59:38.984718 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:59:38.984718 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:59:38.984718 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:59:38.984718 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:59:38.984718 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:59:38.984718 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:59:38.984718 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:59:38.984718 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 18 04:59:38.984718 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 18 04:59:38.984718 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 18 04:59:38.984718 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 04:59:38.990363 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 475/932] 10.222.0.23 () {56 vars in 1132 bytes} [Sat Jun 18 04:59:38 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:59:38.993168 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f76d2d-c521-4d25-8dc0-3ea52607b0d2 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:39.062045 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8af403-dd02-4fc3-9d22-a486e37abe49 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:39.104470 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:39.113428 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663242c7-5203-4970-9c1f-8bd9f7ece0d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:39.113428 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663242c7-5203-4970-9c1f-8bd9f7ece0d2 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:39.113428 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663242c7-5203-4970-9c1f-8bd9f7ece0d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:39.113428 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663242c7-5203-4970-9c1f-8bd9f7ece0d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:39.119999 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f76d2d-c521-4d25-8dc0-3ea52607b0d2 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:39.125996 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663242c7-5203-4970-9c1f-8bd9f7ece0d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:39.127439 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663242c7-5203-4970-9c1f-8bd9f7ece0d2 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:39.148030 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application [None req-663242c7-5203-4970-9c1f-8bd9f7ece0d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 04:59:39.148030 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:39.148030 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:59:39.148030 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:59:39.148030 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:59:39.148030 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:59:39.148030 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:59:39.148030 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:59:39.148030 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:59:39.148030 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:59:39.148030 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:59:39.148030 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:59:39.148030 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 18 04:59:39.148030 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 18 04:59:39.148030 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 18 04:59:39.149001 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 18 04:59:39.149001 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:59:39.149001 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:59:39.149001 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:59:39.149001 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:59:39.149001 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:59:39.149001 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:59:39.149001 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:59:39.149001 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application with Lock( Jun 18 04:59:39.149001 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:59:39.149001 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:59:39.149001 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:59:39.149001 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:59:39.149001 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:59:39.149001 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:59:39.149001 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:59:39.149001 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:59:39.149968 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 18 04:59:39.149968 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 18 04:59:39.149968 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 04:59:39.149968 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 04:59:39.153713 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 458/933] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Jun 18 04:59:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:59:39.155483 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663242c7-5203-4970-9c1f-8bd9f7ece0d2 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:39.157080 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78971a2-a889-4bf1-806a-e140236b7e6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:39.157773 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78971a2-a889-4bf1-806a-e140236b7e6b None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:39.158058 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78971a2-a889-4bf1-806a-e140236b7e6b None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:39.158441 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78971a2-a889-4bf1-806a-e140236b7e6b None admin] PATH_INFO: `/v3/services` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:39.159034 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78971a2-a889-4bf1-806a-e140236b7e6b None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:39.159981 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78971a2-a889-4bf1-806a-e140236b7e6b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:39.160699 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78971a2-a889-4bf1-806a-e140236b7e6b None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:39.166768 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 476/934] 10.222.0.23 () {56 vars in 1126 bytes} [Sat Jun 18 04:59:38 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 18 04:59:39.173207 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78971a2-a889-4bf1-806a-e140236b7e6b None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:39.269079 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:39.287990 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663242c7-5203-4970-9c1f-8bd9f7ece0d2 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:39.304824 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78971a2-a889-4bf1-806a-e140236b7e6b None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:39.340610 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6f9127-9d55-48fe-bd5c-5c660d698193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:39.341910 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6f9127-9d55-48fe-bd5c-5c660d698193 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:39.342784 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6f9127-9d55-48fe-bd5c-5c660d698193 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:39.343548 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6f9127-9d55-48fe-bd5c-5c660d698193 None admin] PATH_INFO: `/v3/domains` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:39.344833 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6f9127-9d55-48fe-bd5c-5c660d698193 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:39.345788 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6f9127-9d55-48fe-bd5c-5c660d698193 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:39.347677 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6f9127-9d55-48fe-bd5c-5c660d698193 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:39.351487 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452c7c4b-51d8-43f4-ae63-2cd6b330797f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:39.352199 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452c7c4b-51d8-43f4-ae63-2cd6b330797f None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:39.352482 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452c7c4b-51d8-43f4-ae63-2cd6b330797f None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:39.352754 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452c7c4b-51d8-43f4-ae63-2cd6b330797f None admin] PATH_INFO: `/v3/services` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:39.353496 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452c7c4b-51d8-43f4-ae63-2cd6b330797f None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:39.353862 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452c7c4b-51d8-43f4-ae63-2cd6b330797f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:39.355229 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452c7c4b-51d8-43f4-ae63-2cd6b330797f None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:39.361143 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 477/935] 10.222.0.23 () {56 vars in 1126 bytes} [Sat Jun 18 04:59:39 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:39.366740 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 459/936] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Jun 18 04:59:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 18 04:59:39.367802 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452c7c4b-51d8-43f4-ae63-2cd6b330797f None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:39.369337 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6f9127-9d55-48fe-bd5c-5c660d698193 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:39.391345 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:39.494900 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6f9127-9d55-48fe-bd5c-5c660d698193 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:39.503554 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452c7c4b-51d8-43f4-ae63-2cd6b330797f None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:39.532476 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0c32f9-add9-41b8-9a49-60847f8c2813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:39.533562 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0c32f9-add9-41b8-9a49-60847f8c2813 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:39.533965 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0c32f9-add9-41b8-9a49-60847f8c2813 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:39.534339 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0c32f9-add9-41b8-9a49-60847f8c2813 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:39.543959 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0c32f9-add9-41b8-9a49-60847f8c2813 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:39.545671 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0c32f9-add9-41b8-9a49-60847f8c2813 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:39.545789 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1981a829-deba-46c8-ae71-db5314cc518c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:39.546556 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1981a829-deba-46c8-ae71-db5314cc518c None admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:39.546844 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1981a829-deba-46c8-ae71-db5314cc518c None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:39.547124 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1981a829-deba-46c8-ae71-db5314cc518c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:39.548038 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1981a829-deba-46c8-ae71-db5314cc518c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:39.549936 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1981a829-deba-46c8-ae71-db5314cc518c None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:39.562368 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application [None req-2c0c32f9-add9-41b8-9a49-60847f8c2813 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 18 04:59:39.562368 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:39.562368 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:59:39.562368 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:59:39.562368 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:59:39.562368 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:59:39.562368 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:59:39.562368 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:59:39.562368 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:59:39.562368 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:59:39.562368 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:59:39.562368 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:59:39.562368 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 18 04:59:39.562368 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 18 04:59:39.562368 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 18 04:59:39.562368 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 18 04:59:39.563563 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:59:39.563563 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:59:39.563563 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 18 04:59:39.563563 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 18 04:59:39.563563 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 18 04:59:39.563563 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 18 04:59:39.563563 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:59:39.563563 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:59:39.563563 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:59:39.563563 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:59:39.563563 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:59:39.563563 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application with Lock( Jun 18 04:59:39.563563 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:59:39.563563 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:59:39.563563 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:59:39.563563 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:59:39.563563 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:59:39.563563 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:59:39.564855 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:59:39.564855 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:59:39.564855 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 18 04:59:39.564855 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 18 04:59:39.564855 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 18 04:59:39.564855 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 18 04:59:39.564855 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 18 04:59:39.564855 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 18 04:59:39.564855 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 18 04:59:39.564855 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 04:59:39.565708 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 460/937] 10.222.0.23 () {56 vars in 1117 bytes} [Sat Jun 18 04:59:39 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:59:39.571287 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0c32f9-add9-41b8-9a49-60847f8c2813 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:39.583218 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:39.664484 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 478/938] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Jun 18 04:59:39 2022] POST /identity/v3/endpoints => generated 347 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 04:59:39.685504 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0c32f9-add9-41b8-9a49-60847f8c2813 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:39.734220 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa414a46-58a3-4462-8998-9e14a3ad0cb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:39.735296 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa414a46-58a3-4462-8998-9e14a3ad0cb8 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:39.735855 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa414a46-58a3-4462-8998-9e14a3ad0cb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:39.736422 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa414a46-58a3-4462-8998-9e14a3ad0cb8 None admin] PATH_INFO: `/v3/users` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:39.737573 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa414a46-58a3-4462-8998-9e14a3ad0cb8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:39.738100 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa414a46-58a3-4462-8998-9e14a3ad0cb8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:39.740321 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa414a46-58a3-4462-8998-9e14a3ad0cb8 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:39.761787 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 461/939] 10.222.0.23 () {56 vars in 1149 bytes} [Sat Jun 18 04:59:39 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 18 04:59:39.770530 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1981a829-deba-46c8-ae71-db5314cc518c None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:39.854805 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:39.884373 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1981a829-deba-46c8-ae71-db5314cc518c None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:39.920603 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80a59c9-6e97-491b-a551-b20ec8ad1dd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:39.921323 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80a59c9-6e97-491b-a551-b20ec8ad1dd8 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:39.921639 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80a59c9-6e97-491b-a551-b20ec8ad1dd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:39.921915 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80a59c9-6e97-491b-a551-b20ec8ad1dd8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:39.933911 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80a59c9-6e97-491b-a551-b20ec8ad1dd8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:39.935370 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80a59c9-6e97-491b-a551-b20ec8ad1dd8 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:39.953038 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-b80a59c9-6e97-491b-a551-b20ec8ad1dd8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 04:59:39.953038 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:39.953038 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:59:39.953038 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:59:39.953038 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:59:39.953038 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:59:39.953038 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:59:39.953038 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:59:39.953038 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:59:39.953038 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:59:39.953038 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:59:39.953038 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:59:39.953038 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 18 04:59:39.953038 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 18 04:59:39.953038 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 18 04:59:39.954014 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 18 04:59:39.954014 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:59:39.954014 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:59:39.954014 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:59:39.954014 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:59:39.954014 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:59:39.954014 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:59:39.954014 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:59:39.954014 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jun 18 04:59:39.954014 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:59:39.954014 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:59:39.954014 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:59:39.954014 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:59:39.954014 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:59:39.954014 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:59:39.954014 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:59:39.954014 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:59:39.954900 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 18 04:59:39.954900 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 18 04:59:39.954900 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 04:59:39.954900 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 04:59:39.954900 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 479/940] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Jun 18 04:59:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:59:39.960771 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa414a46-58a3-4462-8998-9e14a3ad0cb8 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:40.025286 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:40.070097 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa414a46-58a3-4462-8998-9e14a3ad0cb8 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:40.112447 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2505b00c-aabf-49b8-bb51-4485e8d0d709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:40.113108 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2505b00c-aabf-49b8-bb51-4485e8d0d709 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:40.113406 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2505b00c-aabf-49b8-bb51-4485e8d0d709 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:40.113694 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2505b00c-aabf-49b8-bb51-4485e8d0d709 None admin] PATH_INFO: `/v3/domains` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:40.114407 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2505b00c-aabf-49b8-bb51-4485e8d0d709 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:40.114742 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2505b00c-aabf-49b8-bb51-4485e8d0d709 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:40.116540 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2505b00c-aabf-49b8-bb51-4485e8d0d709 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:40.129067 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 462/941] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Jun 18 04:59:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:40.134873 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80a59c9-6e97-491b-a551-b20ec8ad1dd8 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:40.241840 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80a59c9-6e97-491b-a551-b20ec8ad1dd8 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:40.267796 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:40.280527 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b20385-44e3-4d72-9283-c4e36625e34d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:40.281405 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b20385-44e3-4d72-9283-c4e36625e34d None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:40.281689 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b20385-44e3-4d72-9283-c4e36625e34d None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:40.281965 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b20385-44e3-4d72-9283-c4e36625e34d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:40.289561 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b20385-44e3-4d72-9283-c4e36625e34d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:40.291043 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b20385-44e3-4d72-9283-c4e36625e34d None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:40.306230 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-36b20385-44e3-4d72-9283-c4e36625e34d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 18 04:59:40.306230 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:40.306230 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:59:40.306230 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:59:40.306230 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:59:40.306230 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:59:40.306230 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:59:40.306230 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:59:40.306230 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:59:40.306230 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:59:40.306230 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:59:40.306230 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:59:40.306230 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 18 04:59:40.306230 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 18 04:59:40.306230 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 18 04:59:40.307566 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 18 04:59:40.307566 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:59:40.307566 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:59:40.307566 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:59:40.307566 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:59:40.307566 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:59:40.307566 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:59:40.307566 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:59:40.307566 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jun 18 04:59:40.307566 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:59:40.307566 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:59:40.307566 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:59:40.307566 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:59:40.307566 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:59:40.307566 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:59:40.307566 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:59:40.307566 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:59:40.308464 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 18 04:59:40.308464 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 18 04:59:40.308464 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 18 04:59:40.308464 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 18 04:59:40.308464 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 18 04:59:40.308464 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 18 04:59:40.308464 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 18 04:59:40.308464 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 04:59:40.308464 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 480/942] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Jun 18 04:59:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:59:40.313517 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2505b00c-aabf-49b8-bb51-4485e8d0d709 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:40.425651 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a0335f-179b-4a17-8b4f-774440b8cf4f None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:40.426002 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a0335f-179b-4a17-8b4f-774440b8cf4f None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:40.426257 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a0335f-179b-4a17-8b4f-774440b8cf4f None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:40.427934 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 481/943] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 04:59:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 04:59:40.432979 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d92196-9d5a-4808-8a26-38bcaa39291b None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:40.433274 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d92196-9d5a-4808-8a26-38bcaa39291b None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:40.433587 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d92196-9d5a-4808-8a26-38bcaa39291b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:40.442332 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2505b00c-aabf-49b8-bb51-4485e8d0d709 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:40.451157 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-f1d92196-9d5a-4808-8a26-38bcaa39291b None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 04:59:40.452375 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-f1d92196-9d5a-4808-8a26-38bcaa39291b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 04:59:40.468215 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:40.481510 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9773cd2e-508e-4602-a547-b4d00724432c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:40.482503 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9773cd2e-508e-4602-a547-b4d00724432c None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:40.483168 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9773cd2e-508e-4602-a547-b4d00724432c None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:40.483658 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9773cd2e-508e-4602-a547-b4d00724432c None admin] PATH_INFO: `/v3/projects` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:40.484763 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9773cd2e-508e-4602-a547-b4d00724432c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:40.485284 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9773cd2e-508e-4602-a547-b4d00724432c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:40.487002 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9773cd2e-508e-4602-a547-b4d00724432c None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:40.502430 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 463/944] 10.222.0.23 () {56 vars in 1148 bytes} [Sat Jun 18 04:59:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:40.508001 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9773cd2e-508e-4602-a547-b4d00724432c None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:40.527177 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 482/945] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 04:59:40 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 18 04:59:40.551026 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d92196-9d5a-4808-8a26-38bcaa39291b None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:40.631905 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9773cd2e-508e-4602-a547-b4d00724432c None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:40.658910 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d92196-9d5a-4808-8a26-38bcaa39291b None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:40.675529 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bf4485-acee-4798-9b6d-fe44032a3b9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:40.676640 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bf4485-acee-4798-9b6d-fe44032a3b9d None admin] REQUEST_METHOD: `PUT` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:40.676760 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:40.678092 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bf4485-acee-4798-9b6d-fe44032a3b9d None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:40.678269 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bf4485-acee-4798-9b6d-fe44032a3b9d None admin] PATH_INFO: `/v3/projects/c95f8f3084c5472cb8364b7c57207394/users/e817fafd0c1b4bf9944481cf116f1260/roles/2495da35026945aabd29678096996e70` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:40.683130 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bf4485-acee-4798-9b6d-fe44032a3b9d None admin] RBAC: Authorizing `identity:create_grant(project_id=c95f8f3084c5472cb8364b7c57207394, user_id=e817fafd0c1b4bf9944481cf116f1260, role_id=2495da35026945aabd29678096996e70)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:40.685060 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bf4485-acee-4798-9b6d-fe44032a3b9d None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:40.697344 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7d90ee-256e-4e5a-aca3-51f9601cff2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:40.698050 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7d90ee-256e-4e5a-aca3-51f9601cff2b None admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:40.698335 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7d90ee-256e-4e5a-aca3-51f9601cff2b None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:40.698617 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7d90ee-256e-4e5a-aca3-51f9601cff2b None admin] PATH_INFO: `/v3/projects` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:40.699324 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7d90ee-256e-4e5a-aca3-51f9601cff2b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:40.700760 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7d90ee-256e-4e5a-aca3-51f9601cff2b None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:40.702392 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: WARNING keystone.server.flask.common [None req-5f7d90ee-256e-4e5a-aca3-51f9601cff2b None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jun 18 04:59:40.748897 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 464/946] 10.222.0.23 () {58 vars in 1439 bytes} [Sat Jun 18 04:59:40 2022] PUT /identity/v3/projects/c95f8f3084c5472cb8364b7c57207394/users/e817fafd0c1b4bf9944481cf116f1260/roles/2495da35026945aabd29678096996e70 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 04:59:40.764714 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.sql.core [None req-5f7d90ee-256e-4e5a-aca3-51f9601cff2b None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 18 04:59:40.764714 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 18 04:59:40.764714 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [parameters: {'id': '536ef90522204481a481d76411e4712e', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 18 04:59:40.764714 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80469) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 18 04:59:40.773089 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-5f7d90ee-256e-4e5a-aca3-51f9601cff2b None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 536ef90522204481a481d76411e4712e.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 536ef90522204481a481d76411e4712e. Jun 18 04:59:40.773089 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:40.773089 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 18 04:59:40.773089 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 18 04:59:40.773089 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 18 04:59:40.773089 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 18 04:59:40.773089 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 18 04:59:40.773089 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application result = self._query(query) Jun 18 04:59:40.773089 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 18 04:59:40.773089 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application conn.query(q) Jun 18 04:59:40.773089 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 18 04:59:40.773089 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 18 04:59:40.773089 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 18 04:59:40.773996 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application result.read() Jun 18 04:59:40.773996 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 18 04:59:40.773996 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 18 04:59:40.773996 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 18 04:59:40.773996 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 18 04:59:40.773996 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 18 04:59:40.773996 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 18 04:59:40.773996 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 18 04:59:40.773996 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 18 04:59:40.773996 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 18 04:59:40.773996 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 04:59:40.773996 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 18 04:59:40.773996 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 04:59:40.773996 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:40.773996 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 18 04:59:40.773996 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 18 04:59:40.773996 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jun 18 04:59:40.773996 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return project_ref.to_dict() Jun 18 04:59:40.775499 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 18 04:59:40.775499 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application next(self.gen) Jun 18 04:59:40.775499 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 18 04:59:40.775499 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application yield resource Jun 18 04:59:40.775499 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 18 04:59:40.775499 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application next(self.gen) Jun 18 04:59:40.775499 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 18 04:59:40.775499 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application self.session.rollback() Jun 18 04:59:40.775499 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 18 04:59:40.775499 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application self.force_reraise() Jun 18 04:59:40.775499 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 18 04:59:40.775499 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise self.value Jun 18 04:59:40.775499 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 18 04:59:40.775499 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 18 04:59:40.775499 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 18 04:59:40.775499 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application session.commit() Jun 18 04:59:40.775499 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 18 04:59:40.776587 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 18 04:59:40.776587 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 18 04:59:40.776587 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application self._prepare_impl() Jun 18 04:59:40.776587 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 18 04:59:40.776587 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application self.session.flush() Jun 18 04:59:40.776587 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 18 04:59:40.776587 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application self._flush(objects) Jun 18 04:59:40.776587 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 18 04:59:40.776587 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 18 04:59:40.776587 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 18 04:59:40.776587 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application compat.raise_( Jun 18 04:59:40.776587 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 18 04:59:40.776587 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception Jun 18 04:59:40.776587 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 18 04:59:40.776587 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application flush_context.execute() Jun 18 04:59:40.776587 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 18 04:59:40.776587 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rec.execute(self) Jun 18 04:59:40.778231 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 18 04:59:40.778231 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 18 04:59:40.778231 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 18 04:59:40.778231 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 18 04:59:40.778231 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 18 04:59:40.778231 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 18 04:59:40.778231 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 18 04:59:40.778231 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 18 04:59:40.778231 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 18 04:59:40.778231 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 18 04:59:40.778231 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 18 04:59:40.778231 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 18 04:59:40.778231 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 18 04:59:40.778231 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 18 04:59:40.778231 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 18 04:59:40.780186 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 18 04:59:40.780186 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 18 04:59:40.780186 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception Jun 18 04:59:40.780186 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 18 04:59:40.780186 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 18 04:59:40.780186 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 18 04:59:40.780186 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 18 04:59:40.780186 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 18 04:59:40.780186 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application result = self._query(query) Jun 18 04:59:40.780186 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 18 04:59:40.780186 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application conn.query(q) Jun 18 04:59:40.780186 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 18 04:59:40.780186 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 18 04:59:40.780186 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 18 04:59:40.780186 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application result.read() Jun 18 04:59:40.780186 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 18 04:59:40.780186 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 18 04:59:40.784427 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 18 04:59:40.784427 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 18 04:59:40.784427 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 18 04:59:40.784427 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 18 04:59:40.784427 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 18 04:59:40.784427 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 18 04:59:40.784427 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 18 04:59:40.784427 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 18 04:59:40.784427 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [parameters: {'id': '536ef90522204481a481d76411e4712e', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 18 04:59:40.784427 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 18 04:59:40.784427 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 04:59:40.784427 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 18 04:59:40.784427 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 04:59:40.784427 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:40.784427 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jun 18 04:59:40.786373 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jun 18 04:59:40.786373 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 18 04:59:40.786373 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 18 04:59:40.786373 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jun 18 04:59:40.786373 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 04:59:40.786373 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 18 04:59:40.786373 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 04:59:40.786373 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:40.786373 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:59:40.786373 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:59:40.786373 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:59:40.786373 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:59:40.786373 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:59:40.786373 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:59:40.786373 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:59:40.786373 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:59:40.786373 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:59:40.789864 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:59:40.789864 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jun 18 04:59:40.789864 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jun 18 04:59:40.789864 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:59:40.789864 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:59:40.789864 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jun 18 04:59:40.789864 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.Conflict( Jun 18 04:59:40.789864 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 536ef90522204481a481d76411e4712e. Jun 18 04:59:40.789864 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 04:59:40.789864 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 483/947] 10.222.0.23 () {60 vars in 1140 bytes} [Sat Jun 18 04:59:40 2022] POST /identity/v3/projects => generated 262 bytes in 224 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jun 18 04:59:40.791395 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bf4485-acee-4798-9b6d-fe44032a3b9d None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:40.831177 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:40.897324 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bf4485-acee-4798-9b6d-fe44032a3b9d None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:40.935394 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db61d15a-2c0a-4c8b-8ff3-8494557efb42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:40.936197 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db61d15a-2c0a-4c8b-8ff3-8494557efb42 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:40.936481 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db61d15a-2c0a-4c8b-8ff3-8494557efb42 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:40.936761 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db61d15a-2c0a-4c8b-8ff3-8494557efb42 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:40.944679 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db61d15a-2c0a-4c8b-8ff3-8494557efb42 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:40.946235 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db61d15a-2c0a-4c8b-8ff3-8494557efb42 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:40.960764 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application [None req-db61d15a-2c0a-4c8b-8ff3-8494557efb42 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 18 04:59:40.960764 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:40.960764 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:59:40.960764 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:59:40.960764 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:59:40.960764 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:59:40.960764 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:59:40.960764 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:59:40.960764 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:59:40.960764 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:59:40.960764 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:59:40.960764 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:59:40.960764 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 18 04:59:40.960764 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 18 04:59:40.960764 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 18 04:59:40.962907 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 18 04:59:40.962907 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:59:40.962907 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:59:40.962907 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:59:40.962907 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:59:40.962907 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:59:40.962907 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:59:40.962907 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:59:40.962907 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application with Lock( Jun 18 04:59:40.962907 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:59:40.962907 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:59:40.962907 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:59:40.962907 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:59:40.962907 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:59:40.962907 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:59:40.962907 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:59:40.962907 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:59:40.965171 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 18 04:59:40.965171 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 18 04:59:40.965171 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 18 04:59:40.965171 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 18 04:59:40.965171 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 18 04:59:40.965171 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 18 04:59:40.965171 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 18 04:59:40.965171 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 04:59:40.965171 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 465/948] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Jun 18 04:59:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:59:40.968296 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7d90ee-256e-4e5a-aca3-51f9601cff2b None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:41.014251 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b07c71a-4971-4576-9db4-02f1fac29a4a None None] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:41.014933 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b07c71a-4971-4576-9db4-02f1fac29a4a None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:41.015431 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b07c71a-4971-4576-9db4-02f1fac29a4a None None] PATH_INFO: `/` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:41.017950 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 466/949] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 04:59:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 18 04:59:41.024198 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e63df0b-0ccb-4400-8f0f-70c7e6736b2d None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:41.024722 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e63df0b-0ccb-4400-8f0f-70c7e6736b2d None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:41.025197 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e63df0b-0ccb-4400-8f0f-70c7e6736b2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:41.044795 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-9e63df0b-0ccb-4400-8f0f-70c7e6736b2d None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 04:59:41.046472 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-9e63df0b-0ccb-4400-8f0f-70c7e6736b2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 04:59:41.085985 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7d90ee-256e-4e5a-aca3-51f9601cff2b None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:41.121607 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a5e2e8-abe0-4dfc-a83f-d2b2fa861cfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:41.122318 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a5e2e8-abe0-4dfc-a83f-d2b2fa861cfb None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:41.122598 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a5e2e8-abe0-4dfc-a83f-d2b2fa861cfb None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:41.122877 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a5e2e8-abe0-4dfc-a83f-d2b2fa861cfb None admin] PATH_INFO: `/v3/projects` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:41.123814 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a5e2e8-abe0-4dfc-a83f-d2b2fa861cfb None admin] RBAC: Adding query filter params (name=service) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:41.124127 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a5e2e8-abe0-4dfc-a83f-d2b2fa861cfb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:41.126384 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a5e2e8-abe0-4dfc-a83f-d2b2fa861cfb None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:41.145180 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 467/950] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 04:59:41 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 18 04:59:41.156776 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 484/951] 10.222.0.23 () {56 vars in 1112 bytes} [Sat Jun 18 04:59:40 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:41.164188 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e63df0b-0ccb-4400-8f0f-70c7e6736b2d None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:41.203186 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:41.279913 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e63df0b-0ccb-4400-8f0f-70c7e6736b2d None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:41.321033 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cdfc3f-528a-4fdf-8942-7fa6d7bec72e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:41.322337 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cdfc3f-528a-4fdf-8942-7fa6d7bec72e None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:41.322681 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cdfc3f-528a-4fdf-8942-7fa6d7bec72e None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:41.323021 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cdfc3f-528a-4fdf-8942-7fa6d7bec72e None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:41.323735 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cdfc3f-528a-4fdf-8942-7fa6d7bec72e None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:41.325148 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cdfc3f-528a-4fdf-8942-7fa6d7bec72e None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:41.340677 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application [None req-f8cdfc3f-528a-4fdf-8942-7fa6d7bec72e None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 18 04:59:41.340677 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:41.340677 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 18 04:59:41.340677 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 18 04:59:41.340677 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 18 04:59:41.340677 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 18 04:59:41.340677 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 18 04:59:41.340677 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 18 04:59:41.340677 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 18 04:59:41.340677 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 04:59:41.340677 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 18 04:59:41.340677 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 04:59:41.340677 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:41.340677 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:59:41.340677 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:59:41.340677 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:59:41.340677 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:59:41.342211 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:59:41.342211 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:59:41.342211 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:59:41.342211 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:59:41.342211 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:59:41.342211 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:59:41.342211 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 18 04:59:41.342211 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 18 04:59:41.342211 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 18 04:59:41.342211 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 18 04:59:41.342211 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:59:41.342211 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:59:41.342211 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:59:41.342211 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:59:41.342211 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:59:41.342211 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:59:41.343285 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:59:41.343285 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application with Lock( Jun 18 04:59:41.343285 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:59:41.343285 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:59:41.343285 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:59:41.343285 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:59:41.343285 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:59:41.343285 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:59:41.343285 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:59:41.343285 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:59:41.343285 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 18 04:59:41.343285 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 18 04:59:41.343285 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 18 04:59:41.343285 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 04:59:41.345453 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 468/952] 10.222.0.23 () {56 vars in 1126 bytes} [Sat Jun 18 04:59:41 2022] GET /identity/v3/services/object-store => generated 93 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:59:41.351777 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a5e2e8-abe0-4dfc-a83f-d2b2fa861cfb None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:41.400669 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:41.464078 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a5e2e8-abe0-4dfc-a83f-d2b2fa861cfb None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:41.500401 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264ec863-59fb-4322-bb0d-c1af3f74067f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:41.501217 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264ec863-59fb-4322-bb0d-c1af3f74067f None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:41.501624 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264ec863-59fb-4322-bb0d-c1af3f74067f None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:41.502074 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264ec863-59fb-4322-bb0d-c1af3f74067f None admin] PATH_INFO: `/v3/services` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:41.502964 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264ec863-59fb-4322-bb0d-c1af3f74067f None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:41.503401 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264ec863-59fb-4322-bb0d-c1af3f74067f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:41.504868 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264ec863-59fb-4322-bb0d-c1af3f74067f None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:41.510714 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 485/953] 10.222.0.23 () {56 vars in 1122 bytes} [Sat Jun 18 04:59:41 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:41.516504 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cdfc3f-528a-4fdf-8942-7fa6d7bec72e None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:41.619172 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:41.625342 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cdfc3f-528a-4fdf-8942-7fa6d7bec72e None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:41.660104 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95de19c5-ff46-4c7a-bd5a-8d65aca450aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:41.660916 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95de19c5-ff46-4c7a-bd5a-8d65aca450aa None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:41.661344 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95de19c5-ff46-4c7a-bd5a-8d65aca450aa None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:41.662324 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95de19c5-ff46-4c7a-bd5a-8d65aca450aa None admin] PATH_INFO: `/v3/services` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:41.663163 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95de19c5-ff46-4c7a-bd5a-8d65aca450aa None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:41.663575 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95de19c5-ff46-4c7a-bd5a-8d65aca450aa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:41.665440 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95de19c5-ff46-4c7a-bd5a-8d65aca450aa None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:41.671451 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 469/954] 10.222.0.23 () {56 vars in 1122 bytes} [Sat Jun 18 04:59:41 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:41.676765 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264ec863-59fb-4322-bb0d-c1af3f74067f None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:41.781471 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:41.791089 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264ec863-59fb-4322-bb0d-c1af3f74067f None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:41.827099 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d938dda3-283d-4670-8a2b-50852bcf7d0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:41.828192 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d938dda3-283d-4670-8a2b-50852bcf7d0c None admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:41.828679 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d938dda3-283d-4670-8a2b-50852bcf7d0c None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:41.829159 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d938dda3-283d-4670-8a2b-50852bcf7d0c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:41.830136 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d938dda3-283d-4670-8a2b-50852bcf7d0c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:41.831725 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d938dda3-283d-4670-8a2b-50852bcf7d0c None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:41.845159 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63d3505-7cda-411b-b0b1-f9bc5e906d00 None None] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:41.845483 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63d3505-7cda-411b-b0b1-f9bc5e906d00 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:41.845764 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63d3505-7cda-411b-b0b1-f9bc5e906d00 None None] PATH_INFO: `/` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:41.847191 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 470/955] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 04:59:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 18 04:59:41.852715 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09eccc6-0f67-4fd3-bdae-fea1fd49c631 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:41.854514 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09eccc6-0f67-4fd3-bdae-fea1fd49c631 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:41.854514 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09eccc6-0f67-4fd3-bdae-fea1fd49c631 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:41.870969 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-e09eccc6-0f67-4fd3-bdae-fea1fd49c631 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 04:59:41.872262 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-e09eccc6-0f67-4fd3-bdae-fea1fd49c631 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 04:59:41.908640 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 486/956] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Jun 18 04:59:41 2022] POST /identity/v3/endpoints => generated 349 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 04:59:41.962798 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 471/957] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 04:59:41 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 18 04:59:41.986332 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d938dda3-283d-4670-8a2b-50852bcf7d0c None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:42.099796 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d938dda3-283d-4670-8a2b-50852bcf7d0c None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:42.135765 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2b938f-c856-4eed-98a2-d85f9c9c40f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:42.136792 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2b938f-c856-4eed-98a2-d85f9c9c40f3 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:42.137240 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2b938f-c856-4eed-98a2-d85f9c9c40f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:42.137677 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2b938f-c856-4eed-98a2-d85f9c9c40f3 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:42.138523 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2b938f-c856-4eed-98a2-d85f9c9c40f3 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:42.140030 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2b938f-c856-4eed-98a2-d85f9c9c40f3 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:42.152380 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-7a2b938f-c856-4eed-98a2-d85f9c9c40f3 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 18 04:59:42.152380 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:42.152380 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 18 04:59:42.152380 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 18 04:59:42.152380 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 18 04:59:42.152380 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 18 04:59:42.152380 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 18 04:59:42.152380 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 18 04:59:42.152380 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 18 04:59:42.152380 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 04:59:42.152380 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 18 04:59:42.152380 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 04:59:42.152380 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:42.152380 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:59:42.152380 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:59:42.152380 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:59:42.152380 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:59:42.153707 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:59:42.153707 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:59:42.153707 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:59:42.153707 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:59:42.153707 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:59:42.153707 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:59:42.153707 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 18 04:59:42.153707 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 18 04:59:42.153707 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 18 04:59:42.153707 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 18 04:59:42.153707 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:59:42.153707 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:59:42.153707 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:59:42.153707 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:59:42.153707 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:59:42.153707 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:59:42.154763 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:59:42.154763 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jun 18 04:59:42.154763 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:59:42.154763 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:59:42.154763 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:59:42.154763 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:59:42.154763 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:59:42.154763 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:59:42.154763 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:59:42.154763 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:59:42.154763 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 18 04:59:42.154763 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 18 04:59:42.154763 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 18 04:59:42.154763 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 04:59:42.157243 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 487/958] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Jun 18 04:59:41 2022] GET /identity/v3/services/compute => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:59:42.163869 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09eccc6-0f67-4fd3-bdae-fea1fd49c631 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:42.253789 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:42.269830 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09eccc6-0f67-4fd3-bdae-fea1fd49c631 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:42.306992 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0ccdbc-6a5f-422c-b64d-736c19f02c50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:42.307898 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0ccdbc-6a5f-422c-b64d-736c19f02c50 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:42.308357 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0ccdbc-6a5f-422c-b64d-736c19f02c50 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:42.308779 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0ccdbc-6a5f-422c-b64d-736c19f02c50 None admin] PATH_INFO: `/v3/services` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:42.309666 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0ccdbc-6a5f-422c-b64d-736c19f02c50 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:42.310123 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0ccdbc-6a5f-422c-b64d-736c19f02c50 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:42.311616 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0ccdbc-6a5f-422c-b64d-736c19f02c50 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:42.317320 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 472/959] 10.222.0.23 () {56 vars in 1112 bytes} [Sat Jun 18 04:59:42 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:42.323397 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2b938f-c856-4eed-98a2-d85f9c9c40f3 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:42.362270 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:42.435635 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2b938f-c856-4eed-98a2-d85f9c9c40f3 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:42.472932 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da9c80b-37e5-44a7-bcf0-ce35556525c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:42.473900 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da9c80b-37e5-44a7-bcf0-ce35556525c7 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:42.475000 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da9c80b-37e5-44a7-bcf0-ce35556525c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:42.475483 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da9c80b-37e5-44a7-bcf0-ce35556525c7 None admin] PATH_INFO: `/v3/services` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:42.476369 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da9c80b-37e5-44a7-bcf0-ce35556525c7 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:42.476831 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da9c80b-37e5-44a7-bcf0-ce35556525c7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:42.478398 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da9c80b-37e5-44a7-bcf0-ce35556525c7 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:42.483708 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 488/960] 10.222.0.23 () {56 vars in 1112 bytes} [Sat Jun 18 04:59:42 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:42.913407 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f4533a-927d-42f1-980f-31e087dd9e8e None None] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:42.913927 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f4533a-927d-42f1-980f-31e087dd9e8e None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:42.914013 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f4533a-927d-42f1-980f-31e087dd9e8e None None] PATH_INFO: `/` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:42.915585 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 473/961] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 04:59:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 04:59:42.921114 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30972324-25bf-49bc-94cb-b36d575b3c64 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:42.921435 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30972324-25bf-49bc-94cb-b36d575b3c64 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:42.921731 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30972324-25bf-49bc-94cb-b36d575b3c64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:42.939842 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-30972324-25bf-49bc-94cb-b36d575b3c64 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 04:59:42.941400 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-30972324-25bf-49bc-94cb-b36d575b3c64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 04:59:43.013302 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 489/962] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 04:59:42 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 18 04:59:43.039191 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f4533a-927d-42f1-980f-31e087dd9e8e None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:43.148360 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f4533a-927d-42f1-980f-31e087dd9e8e None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:43.183998 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc724c79-8b12-4c8e-8b27-01720cf6ff7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:43.184807 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc724c79-8b12-4c8e-8b27-01720cf6ff7d None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:43.185090 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc724c79-8b12-4c8e-8b27-01720cf6ff7d None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:43.185365 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc724c79-8b12-4c8e-8b27-01720cf6ff7d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:43.196104 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc724c79-8b12-4c8e-8b27-01720cf6ff7d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 18 04:59:43.204250 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application [None req-bc724c79-8b12-4c8e-8b27-01720cf6ff7d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 18 04:59:43.204250 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:43.204250 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:59:43.204250 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:59:43.204250 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:59:43.204250 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:59:43.204250 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:59:43.204250 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:59:43.204250 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:59:43.204250 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:59:43.204250 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:59:43.204250 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:59:43.204250 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 18 04:59:43.204250 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 18 04:59:43.204250 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 18 04:59:43.204250 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 18 04:59:43.205195 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 18 04:59:43.205195 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 18 04:59:43.205195 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 18 04:59:43.205195 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 18 04:59:43.205195 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:59:43.205195 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:59:43.205195 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:59:43.205195 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:59:43.205195 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:59:43.205195 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:59:43.205195 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:59:43.205195 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application with Lock( Jun 18 04:59:43.205195 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:59:43.205195 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:59:43.205195 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:59:43.205195 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:59:43.206071 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:59:43.206071 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:59:43.206071 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:59:43.206071 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:59:43.206071 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 18 04:59:43.206071 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 18 04:59:43.206071 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 18 04:59:43.206071 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 18 04:59:43.206071 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 18 04:59:43.206071 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 18 04:59:43.206071 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 18 04:59:43.206071 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 04:59:43.206071 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 474/963] 10.222.0.23 () {56 vars in 1120 bytes} [Sat Jun 18 04:59:43 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:59:43.212128 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30972324-25bf-49bc-94cb-b36d575b3c64 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:43.224255 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:43.319100 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30972324-25bf-49bc-94cb-b36d575b3c64 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:43.354822 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb3ca55-5622-4751-85cb-f730c7f9ab28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:43.355887 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb3ca55-5622-4751-85cb-f730c7f9ab28 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:43.356410 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb3ca55-5622-4751-85cb-f730c7f9ab28 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:43.356898 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb3ca55-5622-4751-85cb-f730c7f9ab28 None admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:43.357898 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb3ca55-5622-4751-85cb-f730c7f9ab28 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:43.358423 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb3ca55-5622-4751-85cb-f730c7f9ab28 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:43.360001 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb3ca55-5622-4751-85cb-f730c7f9ab28 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:43.369795 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 490/964] 10.222.0.23 () {56 vars in 1115 bytes} [Sat Jun 18 04:59:43 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:43.375984 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc724c79-8b12-4c8e-8b27-01720cf6ff7d None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:43.388698 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cb2275-44c6-459b-8c5c-68c2a1cdef53 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:43.388698 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cb2275-44c6-459b-8c5c-68c2a1cdef53 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:43.388698 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cb2275-44c6-459b-8c5c-68c2a1cdef53 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:43.388698 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 491/965] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 04:59:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 18 04:59:43.393115 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29fb4b9-fcb7-4118-92ed-661b2eea2c6b None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:43.393433 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29fb4b9-fcb7-4118-92ed-661b2eea2c6b None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:43.393720 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29fb4b9-fcb7-4118-92ed-661b2eea2c6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:43.398699 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:43.415146 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-f29fb4b9-fcb7-4118-92ed-661b2eea2c6b None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 04:59:43.417150 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-f29fb4b9-fcb7-4118-92ed-661b2eea2c6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 04:59:43.505880 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 492/966] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 04:59:43 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 18 04:59:43.508248 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc724c79-8b12-4c8e-8b27-01720cf6ff7d None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:43.529729 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29fb4b9-fcb7-4118-92ed-661b2eea2c6b None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:43.532088 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:43.556370 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc20fa98-f6ad-486d-a0b2-2b9eb268e019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:43.557596 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc20fa98-f6ad-486d-a0b2-2b9eb268e019 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:43.558086 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc20fa98-f6ad-486d-a0b2-2b9eb268e019 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:43.559014 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc20fa98-f6ad-486d-a0b2-2b9eb268e019 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:43.575863 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc20fa98-f6ad-486d-a0b2-2b9eb268e019 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:43.577626 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc20fa98-f6ad-486d-a0b2-2b9eb268e019 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:43.603855 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application [None req-cc20fa98-f6ad-486d-a0b2-2b9eb268e019 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 04:59:43.603855 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:43.603855 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:59:43.603855 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:59:43.603855 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:59:43.603855 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:59:43.603855 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:59:43.603855 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:59:43.603855 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:59:43.603855 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:59:43.603855 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:59:43.603855 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:59:43.603855 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 18 04:59:43.603855 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 18 04:59:43.603855 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 18 04:59:43.604796 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 18 04:59:43.604796 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:59:43.604796 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:59:43.604796 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:59:43.604796 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:59:43.604796 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:59:43.604796 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:59:43.604796 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:59:43.604796 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application with Lock( Jun 18 04:59:43.604796 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:59:43.604796 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:59:43.604796 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:59:43.604796 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:59:43.604796 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:59:43.604796 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:59:43.604796 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:59:43.604796 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:59:43.605729 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 18 04:59:43.605729 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 18 04:59:43.605729 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 04:59:43.605729 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 04:59:43.612600 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 475/967] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Jun 18 04:59:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:59:43.613168 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc20fa98-f6ad-486d-a0b2-2b9eb268e019 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:43.659684 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29fb4b9-fcb7-4118-92ed-661b2eea2c6b None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:43.686302 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:43.700156 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eef0936-48f9-42b1-9f0f-bc3acc9bb656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:43.701175 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eef0936-48f9-42b1-9f0f-bc3acc9bb656 None admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:43.706116 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eef0936-48f9-42b1-9f0f-bc3acc9bb656 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:43.706116 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eef0936-48f9-42b1-9f0f-bc3acc9bb656 None admin] PATH_INFO: `/v3/users` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:43.706116 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eef0936-48f9-42b1-9f0f-bc3acc9bb656 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:43.706116 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eef0936-48f9-42b1-9f0f-bc3acc9bb656 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:43.728003 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc20fa98-f6ad-486d-a0b2-2b9eb268e019 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:43.759785 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.sql.core [None req-8eef0936-48f9-42b1-9f0f-bc3acc9bb656 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 18 04:59:43.759785 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 18 04:59:43.759785 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [parameters: {'user_id': '293f49cc920a400784af2f12ff993475', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 18 04:59:43.759785 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80469) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 18 04:59:43.766349 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1f67a5-3f8e-4185-9ac4-b47ddf6345dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:43.767109 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1f67a5-3f8e-4185-9ac4-b47ddf6345dd None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:43.767465 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1f67a5-3f8e-4185-9ac4-b47ddf6345dd None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:43.767814 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1f67a5-3f8e-4185-9ac4-b47ddf6345dd None admin] PATH_INFO: `/v3/domains` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:43.768734 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1f67a5-3f8e-4185-9ac4-b47ddf6345dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:43.769115 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1f67a5-3f8e-4185-9ac4-b47ddf6345dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:43.770591 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1f67a5-3f8e-4185-9ac4-b47ddf6345dd None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:43.771348 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-8eef0936-48f9-42b1-9f0f-bc3acc9bb656 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 18 04:59:43.771348 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:43.771348 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 18 04:59:43.771348 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 18 04:59:43.771348 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 18 04:59:43.771348 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 18 04:59:43.771348 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 18 04:59:43.771348 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application result = self._query(query) Jun 18 04:59:43.771348 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 18 04:59:43.771348 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application conn.query(q) Jun 18 04:59:43.771348 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 18 04:59:43.771348 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 18 04:59:43.771348 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 18 04:59:43.771348 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application result.read() Jun 18 04:59:43.771348 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 18 04:59:43.772227 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 18 04:59:43.772227 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 18 04:59:43.772227 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 18 04:59:43.772227 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 18 04:59:43.772227 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 18 04:59:43.772227 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 18 04:59:43.772227 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 18 04:59:43.772227 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 18 04:59:43.772227 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 04:59:43.772227 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 18 04:59:43.772227 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 04:59:43.772227 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:43.772227 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 18 04:59:43.772227 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 18 04:59:43.772227 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jun 18 04:59:43.772227 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jun 18 04:59:43.772227 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 18 04:59:43.772227 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application next(self.gen) Jun 18 04:59:43.773296 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 18 04:59:43.773296 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application yield resource Jun 18 04:59:43.773296 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 18 04:59:43.773296 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application next(self.gen) Jun 18 04:59:43.773296 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 18 04:59:43.773296 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application self.session.rollback() Jun 18 04:59:43.773296 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 18 04:59:43.773296 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application self.force_reraise() Jun 18 04:59:43.773296 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 18 04:59:43.773296 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise self.value Jun 18 04:59:43.773296 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 18 04:59:43.773296 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 18 04:59:43.773296 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 18 04:59:43.773296 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application session.commit() Jun 18 04:59:43.773296 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 18 04:59:43.773296 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 18 04:59:43.773296 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 18 04:59:43.774338 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application self._prepare_impl() Jun 18 04:59:43.774338 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 18 04:59:43.774338 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application self.session.flush() Jun 18 04:59:43.774338 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 18 04:59:43.774338 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application self._flush(objects) Jun 18 04:59:43.774338 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 18 04:59:43.774338 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 18 04:59:43.774338 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 18 04:59:43.774338 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application compat.raise_( Jun 18 04:59:43.774338 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 18 04:59:43.774338 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception Jun 18 04:59:43.774338 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 18 04:59:43.774338 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application flush_context.execute() Jun 18 04:59:43.774338 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 18 04:59:43.774338 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rec.execute(self) Jun 18 04:59:43.774338 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 18 04:59:43.774338 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 18 04:59:43.775408 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 18 04:59:43.775408 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 18 04:59:43.775408 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jun 18 04:59:43.775408 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application result = connection._execute_20( Jun 18 04:59:43.775408 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 18 04:59:43.775408 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 18 04:59:43.775408 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 18 04:59:43.775408 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 18 04:59:43.775408 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 18 04:59:43.775408 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 18 04:59:43.775408 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 18 04:59:43.775408 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 18 04:59:43.775408 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 18 04:59:43.775408 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 18 04:59:43.775408 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 18 04:59:43.775408 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception Jun 18 04:59:43.776436 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 18 04:59:43.776436 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 18 04:59:43.776436 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 18 04:59:43.776436 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 18 04:59:43.776436 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 18 04:59:43.776436 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application result = self._query(query) Jun 18 04:59:43.776436 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 18 04:59:43.776436 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application conn.query(q) Jun 18 04:59:43.776436 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 18 04:59:43.776436 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 18 04:59:43.776436 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 18 04:59:43.776436 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application result.read() Jun 18 04:59:43.776436 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 18 04:59:43.776436 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 18 04:59:43.776436 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 18 04:59:43.776436 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 18 04:59:43.778798 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 18 04:59:43.778798 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 18 04:59:43.778798 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 18 04:59:43.778798 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 18 04:59:43.778798 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 18 04:59:43.778798 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 18 04:59:43.778798 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [parameters: {'user_id': '293f49cc920a400784af2f12ff993475', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 18 04:59:43.778798 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 18 04:59:43.778798 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 04:59:43.778798 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 18 04:59:43.778798 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 04:59:43.778798 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:43.778798 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:59:43.778798 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:59:43.778798 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:59:43.780824 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:59:43.780824 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:59:43.780824 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:59:43.780824 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:59:43.780824 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:59:43.780824 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:59:43.780824 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:59:43.780824 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jun 18 04:59:43.780824 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jun 18 04:59:43.780824 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:59:43.780824 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:59:43.780824 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 18 04:59:43.780824 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 18 04:59:43.780824 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 18 04:59:43.780824 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 18 04:59:43.780824 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jun 18 04:59:43.780824 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jun 18 04:59:43.783201 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jun 18 04:59:43.783201 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jun 18 04:59:43.783201 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 18 04:59:43.783201 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 18 04:59:43.783201 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 18 04:59:43.783201 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 04:59:43.783201 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 493/968] 10.222.0.23 () {60 vars in 1131 bytes} [Sat Jun 18 04:59:43 2022] POST /identity/v3/users => generated 160 bytes in 252 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jun 18 04:59:43.785135 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eef0936-48f9-42b1-9f0f-bc3acc9bb656 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:43.800925 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 476/969] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Jun 18 04:59:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:43.802729 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1f67a5-3f8e-4185-9ac4-b47ddf6345dd None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:43.854723 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:43.896710 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eef0936-48f9-42b1-9f0f-bc3acc9bb656 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:43.915695 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1f67a5-3f8e-4185-9ac4-b47ddf6345dd None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:43.933340 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6425229-6a84-4cde-9fdf-be90d0fed44f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:43.934126 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6425229-6a84-4cde-9fdf-be90d0fed44f None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:43.934409 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6425229-6a84-4cde-9fdf-be90d0fed44f None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:43.934683 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6425229-6a84-4cde-9fdf-be90d0fed44f None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:43.943257 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:43.943390 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6425229-6a84-4cde-9fdf-be90d0fed44f None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:43.945150 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6425229-6a84-4cde-9fdf-be90d0fed44f None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:43.956105 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a5f0d9-f92f-4a1b-8c5c-91473dc8fda6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:43.957162 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a5f0d9-f92f-4a1b-8c5c-91473dc8fda6 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:43.957894 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a5f0d9-f92f-4a1b-8c5c-91473dc8fda6 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:43.958262 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a5f0d9-f92f-4a1b-8c5c-91473dc8fda6 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:43.960038 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-d6425229-6a84-4cde-9fdf-be90d0fed44f None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 18 04:59:43.960038 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:43.960038 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:59:43.960038 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:59:43.960038 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:59:43.960038 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:59:43.960038 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:59:43.960038 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:59:43.960038 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:59:43.960038 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:59:43.960038 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:59:43.960038 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:59:43.960038 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 18 04:59:43.960038 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 18 04:59:43.960038 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 18 04:59:43.960038 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 18 04:59:43.961120 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:59:43.961120 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:59:43.961120 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 18 04:59:43.961120 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 18 04:59:43.961120 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 18 04:59:43.961120 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 18 04:59:43.961120 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:59:43.961120 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:59:43.961120 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:59:43.961120 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:59:43.961120 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:59:43.961120 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jun 18 04:59:43.961120 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:59:43.961120 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:59:43.961120 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:59:43.961120 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:59:43.961120 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:59:43.961120 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:59:43.962382 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:59:43.962382 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:59:43.962382 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 18 04:59:43.962382 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 18 04:59:43.962382 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 18 04:59:43.962382 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 18 04:59:43.962382 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 18 04:59:43.962382 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 18 04:59:43.962382 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 18 04:59:43.962382 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 04:59:43.962382 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 494/970] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Jun 18 04:59:43 2022] GET /identity/v3/users/cinder => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 18 04:59:43.968643 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6425229-6a84-4cde-9fdf-be90d0fed44f None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:43.972308 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a5f0d9-f92f-4a1b-8c5c-91473dc8fda6 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:43.973782 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a5f0d9-f92f-4a1b-8c5c-91473dc8fda6 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:43.990955 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application [None req-51a5f0d9-f92f-4a1b-8c5c-91473dc8fda6 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 18 04:59:43.990955 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:43.990955 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:59:43.990955 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:59:43.990955 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:59:43.990955 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:59:43.990955 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:59:43.990955 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:59:43.990955 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:59:43.990955 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:59:43.990955 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:59:43.990955 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:59:43.990955 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 18 04:59:43.990955 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 18 04:59:43.990955 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 18 04:59:43.990955 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 18 04:59:43.992207 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:59:43.992207 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:59:43.992207 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 18 04:59:43.992207 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 18 04:59:43.992207 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 18 04:59:43.992207 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 18 04:59:43.992207 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:59:43.992207 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:59:43.992207 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:59:43.992207 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:59:43.992207 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:59:43.992207 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application with Lock( Jun 18 04:59:43.992207 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:59:43.992207 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:59:43.992207 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:59:43.992207 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:59:43.992207 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:59:43.992207 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:59:43.994445 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:59:43.994445 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:59:43.994445 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 18 04:59:43.994445 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 18 04:59:43.994445 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 18 04:59:43.994445 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 18 04:59:43.994445 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 18 04:59:43.994445 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 18 04:59:43.994445 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 18 04:59:43.994445 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 04:59:43.996917 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 477/971] 10.222.0.23 () {56 vars in 1117 bytes} [Sat Jun 18 04:59:43 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:59:43.997958 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a5f0d9-f92f-4a1b-8c5c-91473dc8fda6 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:44.106605 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6425229-6a84-4cde-9fdf-be90d0fed44f None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:44.117037 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a5f0d9-f92f-4a1b-8c5c-91473dc8fda6 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:44.132504 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:44.145592 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08292cdd-0b52-4df7-9871-11180fcd267c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:44.146655 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08292cdd-0b52-4df7-9871-11180fcd267c None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:44.147114 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08292cdd-0b52-4df7-9871-11180fcd267c None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:44.147501 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08292cdd-0b52-4df7-9871-11180fcd267c None admin] PATH_INFO: `/v3/users` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:44.148429 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08292cdd-0b52-4df7-9871-11180fcd267c None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:44.148847 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08292cdd-0b52-4df7-9871-11180fcd267c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:44.150312 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08292cdd-0b52-4df7-9871-11180fcd267c None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:44.155444 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3447dd-eb4c-46cf-9ffb-b0d867642ae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:44.156106 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3447dd-eb4c-46cf-9ffb-b0d867642ae8 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:44.156392 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3447dd-eb4c-46cf-9ffb-b0d867642ae8 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:44.156669 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3447dd-eb4c-46cf-9ffb-b0d867642ae8 None admin] PATH_INFO: `/v3/users` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:44.158149 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3447dd-eb4c-46cf-9ffb-b0d867642ae8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:44.158149 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3447dd-eb4c-46cf-9ffb-b0d867642ae8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:44.159495 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3447dd-eb4c-46cf-9ffb-b0d867642ae8 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:44.178953 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 478/972] 10.222.0.23 () {56 vars in 1149 bytes} [Sat Jun 18 04:59:43 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:44.180249 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774c8fad-7c78-47bb-8e86-368c90cec47c None None] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:44.180540 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774c8fad-7c78-47bb-8e86-368c90cec47c None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:44.180818 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774c8fad-7c78-47bb-8e86-368c90cec47c None None] PATH_INFO: `/` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:44.185490 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 479/973] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 04:59:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 04:59:44.185490 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774c8fad-7c78-47bb-8e86-368c90cec47c None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:44.194164 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 495/974] 10.222.0.23 () {56 vars in 1101 bytes} [Sat Jun 18 04:59:43 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 18 04:59:44.195898 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155106cc-a576-49e7-b559-2590c1dd6840 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:44.196335 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155106cc-a576-49e7-b559-2590c1dd6840 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:44.196748 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155106cc-a576-49e7-b559-2590c1dd6840 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:44.218593 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-155106cc-a576-49e7-b559-2590c1dd6840 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 04:59:44.220192 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-155106cc-a576-49e7-b559-2590c1dd6840 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 04:59:44.247257 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:44.324451 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774c8fad-7c78-47bb-8e86-368c90cec47c None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:44.329571 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 496/975] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 04:59:44 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 18 04:59:44.360394 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155106cc-a576-49e7-b559-2590c1dd6840 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:44.389635 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0575ccfe-a08d-4d8e-a7ac-890af1292c83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:44.390307 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0575ccfe-a08d-4d8e-a7ac-890af1292c83 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:44.392793 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0575ccfe-a08d-4d8e-a7ac-890af1292c83 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:44.393077 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0575ccfe-a08d-4d8e-a7ac-890af1292c83 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:44.484881 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0575ccfe-a08d-4d8e-a7ac-890af1292c83 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:44.486784 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0575ccfe-a08d-4d8e-a7ac-890af1292c83 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:44.501885 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155106cc-a576-49e7-b559-2590c1dd6840 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:44.517349 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:44.535760 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:44.541999 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb36a33a-83c1-433a-b700-0a262b03e151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:44.542793 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb36a33a-83c1-433a-b700-0a262b03e151 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:44.543073 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb36a33a-83c1-433a-b700-0a262b03e151 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:44.543350 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb36a33a-83c1-433a-b700-0a262b03e151 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:44.544039 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb36a33a-83c1-433a-b700-0a262b03e151 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:44.545435 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb36a33a-83c1-433a-b700-0a262b03e151 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:44.558260 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-eb36a33a-83c1-433a-b700-0a262b03e151 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 18 04:59:44.558260 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:44.558260 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 18 04:59:44.558260 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 18 04:59:44.558260 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 18 04:59:44.558260 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 18 04:59:44.558260 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 18 04:59:44.558260 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 18 04:59:44.558260 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 18 04:59:44.558260 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 04:59:44.558260 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 18 04:59:44.558260 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 04:59:44.558260 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:44.558260 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:59:44.558260 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:59:44.558260 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:59:44.558260 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:59:44.559543 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:59:44.559543 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:59:44.559543 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:59:44.559543 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:59:44.559543 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:59:44.559543 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:59:44.559543 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 18 04:59:44.559543 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 18 04:59:44.559543 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 18 04:59:44.559543 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 18 04:59:44.559543 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:59:44.559543 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:59:44.559543 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:59:44.559543 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:59:44.559543 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:59:44.559543 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:59:44.560603 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:59:44.560603 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jun 18 04:59:44.560603 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:59:44.560603 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:59:44.560603 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:59:44.560603 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:59:44.560603 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:59:44.560603 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:59:44.560603 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:59:44.560603 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:59:44.560603 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 18 04:59:44.560603 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 18 04:59:44.560603 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 18 04:59:44.560603 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 04:59:44.568177 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 497/976] 10.222.0.23 () {56 vars in 1126 bytes} [Sat Jun 18 04:59:44 2022] GET /identity/v3/services/object-store => generated 93 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 18 04:59:44.569336 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb36a33a-83c1-433a-b700-0a262b03e151 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:44.602253 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 480/977] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Jun 18 04:59:44 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 18 04:59:44.607813 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0575ccfe-a08d-4d8e-a7ac-890af1292c83 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:44.684539 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb36a33a-83c1-433a-b700-0a262b03e151 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:44.712537 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0575ccfe-a08d-4d8e-a7ac-890af1292c83 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:44.721493 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4603c5a-a8cd-4404-abf0-f7f320c06376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:44.722218 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4603c5a-a8cd-4404-abf0-f7f320c06376 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:44.722501 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4603c5a-a8cd-4404-abf0-f7f320c06376 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:44.722772 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4603c5a-a8cd-4404-abf0-f7f320c06376 None admin] PATH_INFO: `/v3/services` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:44.723476 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4603c5a-a8cd-4404-abf0-f7f320c06376 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:44.723846 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4603c5a-a8cd-4404-abf0-f7f320c06376 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:44.725914 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4603c5a-a8cd-4404-abf0-f7f320c06376 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:44.733572 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 498/978] 10.222.0.23 () {56 vars in 1122 bytes} [Sat Jun 18 04:59:44 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:44.733572 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5deca7-f0de-41e4-a532-7e63667743e6 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:44.733572 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5deca7-f0de-41e4-a532-7e63667743e6 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:44.733572 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5deca7-f0de-41e4-a532-7e63667743e6 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:44.734758 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 499/979] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 04:59:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 04:59:44.736270 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5deca7-f0de-41e4-a532-7e63667743e6 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:44.751312 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfab088-13cd-4442-9e42-7b75166b816f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:44.752095 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfab088-13cd-4442-9e42-7b75166b816f None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:44.752370 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfab088-13cd-4442-9e42-7b75166b816f None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:44.752635 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfab088-13cd-4442-9e42-7b75166b816f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:44.761615 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:44.763673 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfab088-13cd-4442-9e42-7b75166b816f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:44.765097 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfab088-13cd-4442-9e42-7b75166b816f None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:44.780474 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:44.783458 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application [None req-2cfab088-13cd-4442-9e42-7b75166b816f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 04:59:44.783458 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:44.783458 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:59:44.783458 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:59:44.783458 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:59:44.783458 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:59:44.783458 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:59:44.783458 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:59:44.783458 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:59:44.783458 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:59:44.783458 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:59:44.783458 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:59:44.783458 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 18 04:59:44.783458 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 18 04:59:44.783458 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 18 04:59:44.790954 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 18 04:59:44.790954 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:59:44.790954 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:59:44.790954 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:59:44.790954 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:59:44.790954 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:59:44.790954 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:59:44.790954 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:59:44.790954 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application with Lock( Jun 18 04:59:44.790954 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:59:44.790954 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:59:44.790954 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:59:44.790954 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:59:44.790954 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:59:44.790954 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:59:44.790954 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:59:44.790954 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:59:44.791865 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 18 04:59:44.791865 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 18 04:59:44.791865 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 04:59:44.791865 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 04:59:44.791865 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 481/980] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Jun 18 04:59:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:59:44.791865 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635ec29e-733f-4ca5-9043-1f91d7d197b1 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:44.791865 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635ec29e-733f-4ca5-9043-1f91d7d197b1 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:44.791865 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635ec29e-733f-4ca5-9043-1f91d7d197b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:44.804180 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-635ec29e-733f-4ca5-9043-1f91d7d197b1 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 04:59:44.805801 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-635ec29e-733f-4ca5-9043-1f91d7d197b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 04:59:44.840788 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5deca7-f0de-41e4-a532-7e63667743e6 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:44.872648 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 482/981] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 04:59:44 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 18 04:59:44.873499 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635ec29e-733f-4ca5-9043-1f91d7d197b1 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:44.874976 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75d0f32-457e-4cbe-b85f-b4ee0fe9ce0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:44.875628 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75d0f32-457e-4cbe-b85f-b4ee0fe9ce0b None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:44.875913 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75d0f32-457e-4cbe-b85f-b4ee0fe9ce0b None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:44.876188 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75d0f32-457e-4cbe-b85f-b4ee0fe9ce0b None admin] PATH_INFO: `/v3/services` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:44.876885 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75d0f32-457e-4cbe-b85f-b4ee0fe9ce0b None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:44.877196 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75d0f32-457e-4cbe-b85f-b4ee0fe9ce0b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:44.878601 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75d0f32-457e-4cbe-b85f-b4ee0fe9ce0b None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:44.884023 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 500/982] 10.222.0.23 () {56 vars in 1122 bytes} [Sat Jun 18 04:59:44 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 18 04:59:44.889685 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75d0f32-457e-4cbe-b85f-b4ee0fe9ce0b None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:44.982744 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635ec29e-733f-4ca5-9043-1f91d7d197b1 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:44.987570 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75d0f32-457e-4cbe-b85f-b4ee0fe9ce0b None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:44.990436 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:45.007639 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:45.019029 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dff4a07-0202-400d-bb66-23ba1285062b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:45.019699 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dff4a07-0202-400d-bb66-23ba1285062b None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:45.019998 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dff4a07-0202-400d-bb66-23ba1285062b None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:45.020307 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dff4a07-0202-400d-bb66-23ba1285062b None admin] PATH_INFO: `/v3/domains` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:45.021124 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dff4a07-0202-400d-bb66-23ba1285062b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:45.021588 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dff4a07-0202-400d-bb66-23ba1285062b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:45.022831 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dff4a07-0202-400d-bb66-23ba1285062b None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:45.023366 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ded24b5-ff0a-496c-941d-521c2f2d523e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:45.024150 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ded24b5-ff0a-496c-941d-521c2f2d523e None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:45.024327 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ded24b5-ff0a-496c-941d-521c2f2d523e None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:45.024576 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ded24b5-ff0a-496c-941d-521c2f2d523e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:45.025369 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ded24b5-ff0a-496c-941d-521c2f2d523e None admin] RBAC: Adding query filter params (interface=admin, service_id=b4774d3f31b84eba91afda98aa15fad2, region_id=RegionOne) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:45.032185 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ded24b5-ff0a-496c-941d-521c2f2d523e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:45.032185 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ded24b5-ff0a-496c-941d-521c2f2d523e None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:45.032185 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 501/983] 10.222.0.23 () {56 vars in 1249 bytes} [Sat Jun 18 04:59:44 2022] GET /identity/v3/endpoints?service_id=b4774d3f31b84eba91afda98aa15fad2&interface=admin®ion_id=RegionOne => generated 193 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:45.032952 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ded24b5-ff0a-496c-941d-521c2f2d523e None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:45.039877 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 483/984] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Jun 18 04:59:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:45.039877 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dff4a07-0202-400d-bb66-23ba1285062b None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:45.135339 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ded24b5-ff0a-496c-941d-521c2f2d523e None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:45.141241 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dff4a07-0202-400d-bb66-23ba1285062b None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:45.168672 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6794da4f-e44c-4fe5-8e88-aa44e37e4487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:45.169354 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6794da4f-e44c-4fe5-8e88-aa44e37e4487 None admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:45.169677 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6794da4f-e44c-4fe5-8e88-aa44e37e4487 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:45.169959 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6794da4f-e44c-4fe5-8e88-aa44e37e4487 None admin] PATH_INFO: `/v3/services` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:45.170678 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6794da4f-e44c-4fe5-8e88-aa44e37e4487 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:45.172086 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6794da4f-e44c-4fe5-8e88-aa44e37e4487 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:45.173603 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694311df-e643-41f0-b994-1feb2ef829e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:45.174260 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694311df-e643-41f0-b994-1feb2ef829e2 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:45.174541 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694311df-e643-41f0-b994-1feb2ef829e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:45.174810 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694311df-e643-41f0-b994-1feb2ef829e2 None admin] PATH_INFO: `/v3/services` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:45.175493 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694311df-e643-41f0-b994-1feb2ef829e2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:45.176824 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694311df-e643-41f0-b994-1feb2ef829e2 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:45.183242 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 484/985] 10.222.0.23 () {56 vars in 1087 bytes} [Sat Jun 18 04:59:45 2022] GET /identity/v3/services => generated 1497 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 04:59:45.184009 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694311df-e643-41f0-b994-1feb2ef829e2 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:45.236321 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:45.297417 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 502/986] 10.222.0.23 () {60 vars in 1141 bytes} [Sat Jun 18 04:59:45 2022] POST /identity/v3/services => generated 242 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 04:59:45.299619 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694311df-e643-41f0-b994-1feb2ef829e2 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:45.354251 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57c23eb-bd3a-4169-b188-fc1b25c5e567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:45.355079 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57c23eb-bd3a-4169-b188-fc1b25c5e567 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:45.355366 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57c23eb-bd3a-4169-b188-fc1b25c5e567 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:45.356294 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57c23eb-bd3a-4169-b188-fc1b25c5e567 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:45.364180 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57c23eb-bd3a-4169-b188-fc1b25c5e567 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:45.366634 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57c23eb-bd3a-4169-b188-fc1b25c5e567 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:45.386531 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application [None req-b57c23eb-bd3a-4169-b188-fc1b25c5e567 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 18 04:59:45.386531 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:45.386531 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:59:45.386531 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:59:45.386531 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:59:45.386531 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:59:45.386531 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:59:45.386531 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:59:45.386531 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:59:45.386531 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:59:45.386531 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:59:45.386531 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:59:45.386531 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 18 04:59:45.386531 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 18 04:59:45.386531 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 18 04:59:45.387491 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 18 04:59:45.387491 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:59:45.387491 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:59:45.387491 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:59:45.387491 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:59:45.387491 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:59:45.387491 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:59:45.387491 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:59:45.387491 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application with Lock( Jun 18 04:59:45.387491 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:59:45.387491 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:59:45.387491 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:59:45.387491 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:59:45.387491 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:59:45.387491 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:59:45.387491 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:59:45.387491 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:59:45.388561 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 18 04:59:45.388561 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 18 04:59:45.388561 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 18 04:59:45.388561 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 18 04:59:45.388561 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 18 04:59:45.388561 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 18 04:59:45.388561 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 18 04:59:45.388561 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 04:59:45.389368 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 485/987] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Jun 18 04:59:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:59:45.403834 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6794da4f-e44c-4fe5-8e88-aa44e37e4487 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:45.436581 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:45.517165 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6794da4f-e44c-4fe5-8e88-aa44e37e4487 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:45.556182 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8927364-2078-432d-90b9-d2555052089e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:45.556843 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8927364-2078-432d-90b9-d2555052089e None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:45.557126 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8927364-2078-432d-90b9-d2555052089e None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:45.557424 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8927364-2078-432d-90b9-d2555052089e None admin] PATH_INFO: `/v3/projects` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:45.558213 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8927364-2078-432d-90b9-d2555052089e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:45.558545 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8927364-2078-432d-90b9-d2555052089e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:45.559950 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8927364-2078-432d-90b9-d2555052089e None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:45.572081 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 503/988] 10.222.0.23 () {56 vars in 1148 bytes} [Sat Jun 18 04:59:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:45.577820 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57c23eb-bd3a-4169-b188-fc1b25c5e567 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:45.682195 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57c23eb-bd3a-4169-b188-fc1b25c5e567 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:45.683763 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:45.718542 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0250cbd4-9a7f-4633-85f3-af6993ef93cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:45.719197 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0250cbd4-9a7f-4633-85f3-af6993ef93cd None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:45.719478 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0250cbd4-9a7f-4633-85f3-af6993ef93cd None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:45.719750 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0250cbd4-9a7f-4633-85f3-af6993ef93cd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:45.720571 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0250cbd4-9a7f-4633-85f3-af6993ef93cd None admin] RBAC: Adding query filter params (role.id=2495da35026945aabd29678096996e70, scope.project.id=c95f8f3084c5472cb8364b7c57207394, user.id=e817fafd0c1b4bf9944481cf116f1260) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:45.720894 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0250cbd4-9a7f-4633-85f3-af6993ef93cd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:45.722393 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0250cbd4-9a7f-4633-85f3-af6993ef93cd None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:45.729400 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 486/989] 10.222.0.23 () {56 vars in 1374 bytes} [Sat Jun 18 04:59:45 2022] GET /identity/v3/role_assignments?user.id=e817fafd0c1b4bf9944481cf116f1260&scope.project.id=c95f8f3084c5472cb8364b7c57207394&role.id=2495da35026945aabd29678096996e70 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:47.280329 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75899e6-9f20-4f29-83e2-bb98256bef84 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:47.280878 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75899e6-9f20-4f29-83e2-bb98256bef84 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:47.280878 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75899e6-9f20-4f29-83e2-bb98256bef84 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:47.282430 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 504/990] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 04:59:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 04:59:47.287552 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8481961-c129-468e-ab4f-c03097bf8012 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:47.287937 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8481961-c129-468e-ab4f-c03097bf8012 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:47.288122 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8481961-c129-468e-ab4f-c03097bf8012 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:47.305635 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-c8481961-c129-468e-ab4f-c03097bf8012 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 04:59:47.306825 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-c8481961-c129-468e-ab4f-c03097bf8012 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 04:59:47.380076 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 487/991] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 04:59:47 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 18 04:59:47.403803 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75899e6-9f20-4f29-83e2-bb98256bef84 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:47.410963 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e00631-0d56-4020-a260-ef6c36b1f6ea None None] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:47.411256 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e00631-0d56-4020-a260-ef6c36b1f6ea None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:47.411588 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e00631-0d56-4020-a260-ef6c36b1f6ea None None] PATH_INFO: `/` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:47.413786 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 488/992] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 04:59:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 04:59:47.418558 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac570723-6333-45e2-bc52-36103386ecbc None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:47.418850 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac570723-6333-45e2-bc52-36103386ecbc None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:47.419127 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac570723-6333-45e2-bc52-36103386ecbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:47.436635 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-ac570723-6333-45e2-bc52-36103386ecbc None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 04:59:47.437855 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-ac570723-6333-45e2-bc52-36103386ecbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 04:59:47.499467 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:47.508903 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 489/993] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 04:59:47 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 18 04:59:47.512863 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75899e6-9f20-4f29-83e2-bb98256bef84 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:47.531592 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac570723-6333-45e2-bc52-36103386ecbc None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:47.548652 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eeee411-fc49-453a-9d24-bed928c73c75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:47.549497 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eeee411-fc49-453a-9d24-bed928c73c75 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:47.549785 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eeee411-fc49-453a-9d24-bed928c73c75 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:47.550055 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eeee411-fc49-453a-9d24-bed928c73c75 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:47.550707 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eeee411-fc49-453a-9d24-bed928c73c75 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:47.552081 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eeee411-fc49-453a-9d24-bed928c73c75 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:47.560518 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:47.564875 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-7eeee411-fc49-453a-9d24-bed928c73c75 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 18 04:59:47.564875 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:47.564875 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 18 04:59:47.564875 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 18 04:59:47.564875 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 18 04:59:47.564875 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 18 04:59:47.564875 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 18 04:59:47.564875 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 18 04:59:47.564875 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 18 04:59:47.564875 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 04:59:47.564875 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 18 04:59:47.564875 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 04:59:47.564875 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:47.564875 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:59:47.564875 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:59:47.564875 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:59:47.564875 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:59:47.566062 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:59:47.566062 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:59:47.566062 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:59:47.566062 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:59:47.566062 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:59:47.566062 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:59:47.566062 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 18 04:59:47.566062 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 18 04:59:47.566062 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 18 04:59:47.566062 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 18 04:59:47.566062 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:59:47.566062 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:59:47.566062 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:59:47.566062 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:59:47.566062 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:59:47.566062 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:59:47.567020 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:59:47.567020 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jun 18 04:59:47.567020 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:59:47.567020 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:59:47.567020 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:59:47.567020 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:59:47.567020 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:59:47.567020 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:59:47.567020 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:59:47.567020 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:59:47.567020 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 18 04:59:47.567020 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 18 04:59:47.567020 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 18 04:59:47.567020 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 04:59:47.567020 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 505/994] 10.222.0.23 () {56 vars in 1126 bytes} [Sat Jun 18 04:59:47 2022] GET /identity/v3/services/object-store => generated 93 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:59:47.572056 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eeee411-fc49-453a-9d24-bed928c73c75 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:47.642465 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac570723-6333-45e2-bc52-36103386ecbc None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:47.678591 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b6640e-cb95-4c1e-b34d-c3739166ebb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:47.679401 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b6640e-cb95-4c1e-b34d-c3739166ebb8 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:47.679661 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b6640e-cb95-4c1e-b34d-c3739166ebb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:47.679939 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b6640e-cb95-4c1e-b34d-c3739166ebb8 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:47.680585 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b6640e-cb95-4c1e-b34d-c3739166ebb8 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:47.681982 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b6640e-cb95-4c1e-b34d-c3739166ebb8 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:47.683471 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eeee411-fc49-453a-9d24-bed928c73c75 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:47.694485 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application [None req-a3b6640e-cb95-4c1e-b34d-c3739166ebb8 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 18 04:59:47.694485 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:47.694485 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 18 04:59:47.694485 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 18 04:59:47.694485 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 18 04:59:47.694485 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 18 04:59:47.694485 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 18 04:59:47.694485 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 18 04:59:47.694485 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 18 04:59:47.694485 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 04:59:47.694485 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 18 04:59:47.694485 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 04:59:47.694485 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:47.694485 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:59:47.694485 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:59:47.694485 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:59:47.694485 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:59:47.697240 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:59:47.697240 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:59:47.697240 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:59:47.697240 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:59:47.697240 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:59:47.697240 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:59:47.697240 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 18 04:59:47.697240 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 18 04:59:47.697240 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 18 04:59:47.697240 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 18 04:59:47.697240 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:59:47.697240 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:59:47.697240 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:59:47.697240 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:59:47.697240 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:59:47.697240 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:59:47.698342 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:59:47.698342 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application with Lock( Jun 18 04:59:47.698342 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:59:47.698342 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:59:47.698342 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:59:47.698342 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:59:47.698342 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:59:47.698342 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:59:47.698342 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:59:47.698342 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:59:47.698342 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 18 04:59:47.698342 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 18 04:59:47.698342 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 18 04:59:47.698342 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 04:59:47.698342 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 490/995] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Jun 18 04:59:47 2022] GET /identity/v3/services/compute => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 18 04:59:47.701476 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b6640e-cb95-4c1e-b34d-c3739166ebb8 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:47.730613 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3519dda-6d2e-432d-9f51-d99b1436a35d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:47.731417 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3519dda-6d2e-432d-9f51-d99b1436a35d None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:47.731706 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3519dda-6d2e-432d-9f51-d99b1436a35d None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:47.731991 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3519dda-6d2e-432d-9f51-d99b1436a35d None admin] PATH_INFO: `/v3/services` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:47.732697 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3519dda-6d2e-432d-9f51-d99b1436a35d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:47.733008 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3519dda-6d2e-432d-9f51-d99b1436a35d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:47.734433 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3519dda-6d2e-432d-9f51-d99b1436a35d None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:47.739632 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 506/996] 10.222.0.23 () {56 vars in 1122 bytes} [Sat Jun 18 04:59:47 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 18 04:59:47.745533 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3519dda-6d2e-432d-9f51-d99b1436a35d None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:47.766050 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:47.807700 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:47.817509 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b6640e-cb95-4c1e-b34d-c3739166ebb8 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:47.851259 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3519dda-6d2e-432d-9f51-d99b1436a35d None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:47.853877 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6180003b-e2cb-46bb-9a36-e987550a3ed0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:47.854537 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6180003b-e2cb-46bb-9a36-e987550a3ed0 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:47.854823 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6180003b-e2cb-46bb-9a36-e987550a3ed0 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:47.855100 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6180003b-e2cb-46bb-9a36-e987550a3ed0 None admin] PATH_INFO: `/v3/services` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:47.855803 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6180003b-e2cb-46bb-9a36-e987550a3ed0 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:47.856122 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6180003b-e2cb-46bb-9a36-e987550a3ed0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:47.857503 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6180003b-e2cb-46bb-9a36-e987550a3ed0 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:47.862259 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 491/997] 10.222.0.23 () {56 vars in 1112 bytes} [Sat Jun 18 04:59:47 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 18 04:59:47.863444 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b54568-7f90-456e-9684-b84045456da1 None None] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:47.863733 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b54568-7f90-456e-9684-b84045456da1 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:47.864008 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b54568-7f90-456e-9684-b84045456da1 None None] PATH_INFO: `/` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:47.866010 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 492/998] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 04:59:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 04:59:47.871428 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295b1581-4168-48c4-b10d-ede52d7a9942 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:47.871719 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295b1581-4168-48c4-b10d-ede52d7a9942 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:47.872001 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295b1581-4168-48c4-b10d-ede52d7a9942 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:47.886803 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf74f799-ef54-4bd7-9f3a-0e886de4f963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:47.887457 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf74f799-ef54-4bd7-9f3a-0e886de4f963 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:47.887738 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf74f799-ef54-4bd7-9f3a-0e886de4f963 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:47.888013 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf74f799-ef54-4bd7-9f3a-0e886de4f963 None admin] PATH_INFO: `/v3/services` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:47.888464 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-295b1581-4168-48c4-b10d-ede52d7a9942 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 04:59:47.888705 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf74f799-ef54-4bd7-9f3a-0e886de4f963 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:47.889013 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf74f799-ef54-4bd7-9f3a-0e886de4f963 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:47.889688 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-295b1581-4168-48c4-b10d-ede52d7a9942 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 04:59:47.890401 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf74f799-ef54-4bd7-9f3a-0e886de4f963 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:47.895809 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 507/999] 10.222.0.23 () {56 vars in 1122 bytes} [Sat Jun 18 04:59:47 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:47.895809 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf74f799-ef54-4bd7-9f3a-0e886de4f963 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:47.956427 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 493/1000] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 04:59:47 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 18 04:59:47.957459 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295b1581-4168-48c4-b10d-ede52d7a9942 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:47.994711 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:48.013371 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf74f799-ef54-4bd7-9f3a-0e886de4f963 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:48.040616 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:48.047340 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f952f12-cab8-4449-85df-ef79e23c6f09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:48.047995 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f952f12-cab8-4449-85df-ef79e23c6f09 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:48.048295 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f952f12-cab8-4449-85df-ef79e23c6f09 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:48.048559 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f952f12-cab8-4449-85df-ef79e23c6f09 None admin] PATH_INFO: `/v3/services` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:48.049262 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f952f12-cab8-4449-85df-ef79e23c6f09 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:48.051390 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f952f12-cab8-4449-85df-ef79e23c6f09 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:48.051390 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f952f12-cab8-4449-85df-ef79e23c6f09 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:48.056020 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 508/1001] 10.222.0.23 () {56 vars in 1112 bytes} [Sat Jun 18 04:59:47 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:48.056634 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f952f12-cab8-4449-85df-ef79e23c6f09 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:48.056733 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295b1581-4168-48c4-b10d-ede52d7a9942 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:48.091932 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61d297c-1f15-4205-9c8a-ff2f1a9e49f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:48.092519 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61d297c-1f15-4205-9c8a-ff2f1a9e49f5 None admin] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:48.092808 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61d297c-1f15-4205-9c8a-ff2f1a9e49f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:48.093092 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61d297c-1f15-4205-9c8a-ff2f1a9e49f5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:48.093869 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61d297c-1f15-4205-9c8a-ff2f1a9e49f5 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:48.095287 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61d297c-1f15-4205-9c8a-ff2f1a9e49f5 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:48.156662 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f952f12-cab8-4449-85df-ef79e23c6f09 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:48.189580 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d1a27d-2fd8-49c9-baa5-203cf7005471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:48.190355 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d1a27d-2fd8-49c9-baa5-203cf7005471 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:48.190634 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d1a27d-2fd8-49c9-baa5-203cf7005471 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:48.190904 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d1a27d-2fd8-49c9-baa5-203cf7005471 None admin] PATH_INFO: `/v3/services/image` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:48.191549 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d1a27d-2fd8-49c9-baa5-203cf7005471 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:48.193590 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d1a27d-2fd8-49c9-baa5-203cf7005471 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:48.197487 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 494/1002] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Jun 18 04:59:47 2022] POST /identity/v3/endpoints => generated 325 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 04:59:48.198299 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61d297c-1f15-4205-9c8a-ff2f1a9e49f5 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:48.205167 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-d9d1a27d-2fd8-49c9-baa5-203cf7005471 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 18 04:59:48.205167 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:48.205167 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 18 04:59:48.205167 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 18 04:59:48.205167 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 18 04:59:48.205167 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 18 04:59:48.205167 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 18 04:59:48.205167 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 18 04:59:48.205167 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 18 04:59:48.205167 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 04:59:48.205167 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 18 04:59:48.205167 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 04:59:48.205167 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:48.205167 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:59:48.205167 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:59:48.205167 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:59:48.205167 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:59:48.206343 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:59:48.206343 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:59:48.206343 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:59:48.206343 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:59:48.206343 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:59:48.206343 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:59:48.206343 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 18 04:59:48.206343 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 18 04:59:48.206343 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 18 04:59:48.206343 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 18 04:59:48.206343 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:59:48.206343 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:59:48.206343 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:59:48.206343 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:59:48.206343 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:59:48.206343 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:59:48.213111 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:59:48.213111 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jun 18 04:59:48.213111 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:59:48.213111 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:59:48.213111 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:59:48.213111 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:59:48.213111 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:59:48.213111 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:59:48.213111 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:59:48.213111 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:59:48.213111 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 18 04:59:48.213111 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 18 04:59:48.213111 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 18 04:59:48.213111 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 04:59:48.213111 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 509/1003] 10.222.0.23 () {56 vars in 1105 bytes} [Sat Jun 18 04:59:48 2022] GET /identity/v3/services/image => generated 86 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:59:48.213993 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d1a27d-2fd8-49c9-baa5-203cf7005471 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:48.225284 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:48.327352 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d1a27d-2fd8-49c9-baa5-203cf7005471 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:48.328955 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61d297c-1f15-4205-9c8a-ff2f1a9e49f5 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:48.368106 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d826efbf-1554-42b5-9c2c-0893829cc0e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:48.368785 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d826efbf-1554-42b5-9c2c-0893829cc0e9 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:48.369073 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d826efbf-1554-42b5-9c2c-0893829cc0e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:48.369185 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543017b5-5ee3-4473-a385-3e9c067c93f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:48.369360 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d826efbf-1554-42b5-9c2c-0893829cc0e9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:48.369860 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543017b5-5ee3-4473-a385-3e9c067c93f5 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:48.370142 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543017b5-5ee3-4473-a385-3e9c067c93f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:48.370243 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d826efbf-1554-42b5-9c2c-0893829cc0e9 None admin] RBAC: Adding query filter params (interface=public, service_id=1838f2e7727f462a8cb0828a7be1bce8, region_id=RegionOne) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:48.370411 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543017b5-5ee3-4473-a385-3e9c067c93f5 None admin] PATH_INFO: `/v3/services` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:48.370515 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d826efbf-1554-42b5-9c2c-0893829cc0e9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:48.371101 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543017b5-5ee3-4473-a385-3e9c067c93f5 None admin] RBAC: Adding query filter params (name=image) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:48.371408 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543017b5-5ee3-4473-a385-3e9c067c93f5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:48.371853 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d826efbf-1554-42b5-9c2c-0893829cc0e9 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:48.372747 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543017b5-5ee3-4473-a385-3e9c067c93f5 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:48.378134 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 495/1004] 10.222.0.23 () {56 vars in 1251 bytes} [Sat Jun 18 04:59:48 2022] GET /identity/v3/endpoints?service_id=1838f2e7727f462a8cb0828a7be1bce8&interface=public®ion_id=RegionOne => generated 194 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:48.384113 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 510/1005] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Jun 18 04:59:48 2022] GET /identity/v3/services?name=image => generated 122 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 18 04:59:48.384729 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d826efbf-1554-42b5-9c2c-0893829cc0e9 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:48.389410 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543017b5-5ee3-4473-a385-3e9c067c93f5 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:48.389719 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:48.464688 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:48.496972 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543017b5-5ee3-4473-a385-3e9c067c93f5 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:48.504782 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d826efbf-1554-42b5-9c2c-0893829cc0e9 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:48.534475 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dd8dba-ceb0-4981-bd61-4739cdbbf100 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:48.535131 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dd8dba-ceb0-4981-bd61-4739cdbbf100 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:48.535649 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dd8dba-ceb0-4981-bd61-4739cdbbf100 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:48.535649 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dd8dba-ceb0-4981-bd61-4739cdbbf100 None admin] PATH_INFO: `/v3/services` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:48.536365 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dd8dba-ceb0-4981-bd61-4739cdbbf100 None admin] RBAC: Adding query filter params (type=image) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:48.537161 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dd8dba-ceb0-4981-bd61-4739cdbbf100 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:48.539144 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dd8dba-ceb0-4981-bd61-4739cdbbf100 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:48.541445 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9073c75-71bc-41c7-adcb-970058bed0c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:48.542077 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9073c75-71bc-41c7-adcb-970058bed0c4 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:48.542450 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9073c75-71bc-41c7-adcb-970058bed0c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:48.542630 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9073c75-71bc-41c7-adcb-970058bed0c4 None admin] PATH_INFO: `/v3/services` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:48.543313 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9073c75-71bc-41c7-adcb-970058bed0c4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:48.544837 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9073c75-71bc-41c7-adcb-970058bed0c4 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:48.544954 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 511/1006] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Jun 18 04:59:48 2022] GET /identity/v3/services?type=image => generated 122 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:48.561779 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 496/1007] 10.222.0.23 () {56 vars in 1087 bytes} [Sat Jun 18 04:59:48 2022] GET /identity/v3/services => generated 1727 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 04:59:50.281401 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69f3654-9078-4ef8-9b83-33c76df1f0ca None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:50.281924 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69f3654-9078-4ef8-9b83-33c76df1f0ca None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:50.282000 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69f3654-9078-4ef8-9b83-33c76df1f0ca None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:50.283516 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 512/1008] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 04:59:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 04:59:50.288813 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c60a40-323b-4a5f-b3ba-23069dc67cbb None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:50.289235 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c60a40-323b-4a5f-b3ba-23069dc67cbb None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:50.289460 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c60a40-323b-4a5f-b3ba-23069dc67cbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:50.306608 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-d0c60a40-323b-4a5f-b3ba-23069dc67cbb None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 04:59:50.307805 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-d0c60a40-323b-4a5f-b3ba-23069dc67cbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 04:59:50.372857 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:50.383611 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 497/1009] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 04:59:50 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 18 04:59:50.407607 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69f3654-9078-4ef8-9b83-33c76df1f0ca None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:50.514408 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69f3654-9078-4ef8-9b83-33c76df1f0ca None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:50.549468 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c2b33e-5d02-4dbc-8ea1-40788948688f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:50.550248 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c2b33e-5d02-4dbc-8ea1-40788948688f None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:50.550538 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c2b33e-5d02-4dbc-8ea1-40788948688f None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:50.550817 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c2b33e-5d02-4dbc-8ea1-40788948688f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:50.552000 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c2b33e-5d02-4dbc-8ea1-40788948688f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:50.553991 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c2b33e-5d02-4dbc-8ea1-40788948688f None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:50.560516 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 513/1010] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Jun 18 04:59:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:50.561714 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c60a40-323b-4a5f-b3ba-23069dc67cbb None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:50.612977 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcda0acb-93f5-461f-b8ea-c944a9dbfafa None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:50.613265 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcda0acb-93f5-461f-b8ea-c944a9dbfafa None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:50.613614 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcda0acb-93f5-461f-b8ea-c944a9dbfafa None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:50.615627 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 514/1011] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 04:59:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 18 04:59:50.620760 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806508a3-8775-4b60-816b-0cbc753d343d None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:50.621050 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806508a3-8775-4b60-816b-0cbc753d343d None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:50.621332 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806508a3-8775-4b60-816b-0cbc753d343d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:50.628515 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:50.638421 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-806508a3-8775-4b60-816b-0cbc753d343d None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 04:59:50.642239 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-806508a3-8775-4b60-816b-0cbc753d343d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 04:59:50.672641 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c60a40-323b-4a5f-b3ba-23069dc67cbb None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:50.708558 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ed8b40-f7c0-496d-933c-0584b3e521a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:50.708937 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 515/1012] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 04:59:50 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 18 04:59:50.709308 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ed8b40-f7c0-496d-933c-0584b3e521a3 None admin] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:50.709619 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ed8b40-f7c0-496d-933c-0584b3e521a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:50.709886 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ed8b40-f7c0-496d-933c-0584b3e521a3 None admin] PATH_INFO: `/v3/projects` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:50.710388 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46aecf02-2594-48a5-91f1-f62149338e18 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:50.710673 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46aecf02-2594-48a5-91f1-f62149338e18 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:50.710779 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ed8b40-f7c0-496d-933c-0584b3e521a3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:50.710952 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46aecf02-2594-48a5-91f1-f62149338e18 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:50.712152 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ed8b40-f7c0-496d-933c-0584b3e521a3 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:50.712256 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 516/1013] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 04:59:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 04:59:50.717557 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce50533-3570-4cfe-829d-5cabc59afa45 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:50.717818 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce50533-3570-4cfe-829d-5cabc59afa45 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:50.718097 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce50533-3570-4cfe-829d-5cabc59afa45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:50.736789 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-7ce50533-3570-4cfe-829d-5cabc59afa45 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 04:59:50.738050 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-7ce50533-3570-4cfe-829d-5cabc59afa45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 04:59:50.804867 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 517/1014] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 04:59:50 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 18 04:59:50.805626 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce50533-3570-4cfe-829d-5cabc59afa45 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:50.818465 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 498/1015] 10.222.0.23 () {60 vars in 1141 bytes} [Sat Jun 18 04:59:50 2022] POST /identity/v3/projects => generated 311 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 04:59:50.828930 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ed8b40-f7c0-496d-933c-0584b3e521a3 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:50.880187 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:50.888063 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:50.923477 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce50533-3570-4cfe-829d-5cabc59afa45 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:50.945413 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ed8b40-f7c0-496d-933c-0584b3e521a3 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:50.963032 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034f539b-0cff-43c3-bff3-bab1f6605c37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:50.963709 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034f539b-0cff-43c3-bff3-bab1f6605c37 None admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:50.963994 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034f539b-0cff-43c3-bff3-bab1f6605c37 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:50.964274 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034f539b-0cff-43c3-bff3-bab1f6605c37 None admin] PATH_INFO: `/v3/services` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:50.965052 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034f539b-0cff-43c3-bff3-bab1f6605c37 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:50.966464 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034f539b-0cff-43c3-bff3-bab1f6605c37 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:50.983884 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1aec8a0-0071-4f11-92ad-53f1282b8961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:50.984692 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aec8a0-0071-4f11-92ad-53f1282b8961 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:50.984957 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aec8a0-0071-4f11-92ad-53f1282b8961 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:50.985285 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aec8a0-0071-4f11-92ad-53f1282b8961 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:50.985915 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1aec8a0-0071-4f11-92ad-53f1282b8961 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:50.987280 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1aec8a0-0071-4f11-92ad-53f1282b8961 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:51.001516 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application [None req-d1aec8a0-0071-4f11-92ad-53f1282b8961 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 18 04:59:51.001516 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:51.001516 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 18 04:59:51.001516 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 18 04:59:51.001516 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 18 04:59:51.001516 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 18 04:59:51.001516 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 18 04:59:51.001516 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 18 04:59:51.001516 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 18 04:59:51.001516 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 04:59:51.001516 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 18 04:59:51.001516 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 04:59:51.001516 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:51.001516 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:59:51.001516 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:59:51.001516 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:59:51.001516 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:59:51.002729 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:59:51.002729 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:59:51.002729 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:59:51.002729 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:59:51.002729 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:59:51.002729 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:59:51.002729 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 18 04:59:51.002729 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 18 04:59:51.002729 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 18 04:59:51.002729 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 18 04:59:51.002729 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:59:51.002729 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:59:51.002729 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:59:51.002729 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:59:51.002729 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:59:51.002729 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:59:51.003679 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:59:51.003679 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application with Lock( Jun 18 04:59:51.003679 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:59:51.003679 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:59:51.003679 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:59:51.003679 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:59:51.003679 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:59:51.003679 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:59:51.003679 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:59:51.003679 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:59:51.003679 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 18 04:59:51.003679 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 18 04:59:51.003679 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 18 04:59:51.003679 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 04:59:51.003679 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 499/1016] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Jun 18 04:59:50 2022] GET /identity/v3/services/compute => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 18 04:59:51.010420 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1aec8a0-0071-4f11-92ad-53f1282b8961 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:51.093436 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 518/1017] 10.222.0.23 () {60 vars in 1141 bytes} [Sat Jun 18 04:59:50 2022] POST /identity/v3/services => generated 242 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 04:59:51.123183 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1aec8a0-0071-4f11-92ad-53f1282b8961 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:51.139476 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:51.169497 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e85523f-7888-4730-9da1-7d59d87aade4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:51.170176 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e85523f-7888-4730-9da1-7d59d87aade4 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:51.170462 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e85523f-7888-4730-9da1-7d59d87aade4 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:51.170741 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e85523f-7888-4730-9da1-7d59d87aade4 None admin] PATH_INFO: `/v3/services` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:51.171464 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e85523f-7888-4730-9da1-7d59d87aade4 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:51.171779 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e85523f-7888-4730-9da1-7d59d87aade4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:51.173176 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e85523f-7888-4730-9da1-7d59d87aade4 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:51.180397 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 500/1018] 10.222.0.23 () {56 vars in 1112 bytes} [Sat Jun 18 04:59:51 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 18 04:59:51.186162 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034f539b-0cff-43c3-bff3-bab1f6605c37 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:51.301541 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034f539b-0cff-43c3-bff3-bab1f6605c37 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:51.326818 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:51.337983 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbb6bfc-362d-4d1a-aa03-16b3dbb60b93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:51.340225 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbb6bfc-362d-4d1a-aa03-16b3dbb60b93 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:51.340225 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbb6bfc-362d-4d1a-aa03-16b3dbb60b93 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:51.340225 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbb6bfc-362d-4d1a-aa03-16b3dbb60b93 None admin] PATH_INFO: `/v3/services` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:51.340225 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbb6bfc-362d-4d1a-aa03-16b3dbb60b93 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:51.340543 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbb6bfc-362d-4d1a-aa03-16b3dbb60b93 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:51.341831 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbb6bfc-362d-4d1a-aa03-16b3dbb60b93 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:51.349983 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 519/1019] 10.222.0.23 () {56 vars in 1112 bytes} [Sat Jun 18 04:59:51 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:51.353267 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e85523f-7888-4730-9da1-7d59d87aade4 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:51.468243 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e85523f-7888-4730-9da1-7d59d87aade4 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:51.502622 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ba9403-968e-416f-9cfb-8f1f79b46e45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:51.503274 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ba9403-968e-416f-9cfb-8f1f79b46e45 None admin] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:51.503556 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ba9403-968e-416f-9cfb-8f1f79b46e45 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:51.503828 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ba9403-968e-416f-9cfb-8f1f79b46e45 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:51.504567 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ba9403-968e-416f-9cfb-8f1f79b46e45 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:51.506689 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ba9403-968e-416f-9cfb-8f1f79b46e45 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:51.609410 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 501/1020] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Jun 18 04:59:51 2022] POST /identity/v3/endpoints => generated 334 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 04:59:52.926820 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cdaeb3-cf09-4b8f-a4f8-5a4277208996 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:52.927452 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cdaeb3-cf09-4b8f-a4f8-5a4277208996 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:52.927452 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cdaeb3-cf09-4b8f-a4f8-5a4277208996 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:52.928994 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 520/1021] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 04:59:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 04:59:52.933971 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a30351d-a5f8-4fd0-af69-16343919dce4 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:52.934382 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a30351d-a5f8-4fd0-af69-16343919dce4 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:52.934546 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a30351d-a5f8-4fd0-af69-16343919dce4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:52.952475 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-9a30351d-a5f8-4fd0-af69-16343919dce4 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 04:59:52.953692 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-9a30351d-a5f8-4fd0-af69-16343919dce4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 04:59:52.979106 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:53.029554 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 502/1022] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 04:59:52 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 18 04:59:53.055023 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cdaeb3-cf09-4b8f-a4f8-5a4277208996 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:53.160422 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cdaeb3-cf09-4b8f-a4f8-5a4277208996 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:53.164094 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7332b13-597a-414a-94da-8084425cb920 None None] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:53.164384 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7332b13-597a-414a-94da-8084425cb920 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:53.164665 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7332b13-597a-414a-94da-8084425cb920 None None] PATH_INFO: `/` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:53.166072 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 503/1023] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 04:59:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 04:59:53.170995 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82dc189-1558-4657-8d7d-535438ce5e27 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:53.171296 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82dc189-1558-4657-8d7d-535438ce5e27 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:53.171573 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82dc189-1558-4657-8d7d-535438ce5e27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:53.188878 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-f82dc189-1558-4657-8d7d-535438ce5e27 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 04:59:53.190021 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-f82dc189-1558-4657-8d7d-535438ce5e27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 04:59:53.197906 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebf9f20-9985-486a-a2ed-2689ef81bc7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:53.198916 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebf9f20-9985-486a-a2ed-2689ef81bc7b None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:53.199379 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebf9f20-9985-486a-a2ed-2689ef81bc7b None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:53.199864 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebf9f20-9985-486a-a2ed-2689ef81bc7b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:53.201217 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebf9f20-9985-486a-a2ed-2689ef81bc7b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:53.202863 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebf9f20-9985-486a-a2ed-2689ef81bc7b None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:53.204672 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 521/1024] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Jun 18 04:59:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:53.210735 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebf9f20-9985-486a-a2ed-2689ef81bc7b None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:53.262648 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 504/1025] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 04:59:53 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 18 04:59:53.275492 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:53.286528 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82dc189-1558-4657-8d7d-535438ce5e27 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:53.325506 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebf9f20-9985-486a-a2ed-2689ef81bc7b None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:53.363210 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38f9d2e-52fd-4d23-bd7a-fc427751de54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:53.364243 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38f9d2e-52fd-4d23-bd7a-fc427751de54 None admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:53.364745 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38f9d2e-52fd-4d23-bd7a-fc427751de54 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:53.365264 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38f9d2e-52fd-4d23-bd7a-fc427751de54 None admin] PATH_INFO: `/v3/users` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:53.366346 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38f9d2e-52fd-4d23-bd7a-fc427751de54 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:53.368217 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38f9d2e-52fd-4d23-bd7a-fc427751de54 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:53.381650 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:53.400371 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82dc189-1558-4657-8d7d-535438ce5e27 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:53.434043 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252e467f-3ec2-4b4b-9532-6f01faa57fb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:53.434820 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252e467f-3ec2-4b4b-9532-6f01faa57fb0 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:53.435098 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252e467f-3ec2-4b4b-9532-6f01faa57fb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:53.435366 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252e467f-3ec2-4b4b-9532-6f01faa57fb0 None admin] PATH_INFO: `/v3/services/image` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:53.436017 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252e467f-3ec2-4b4b-9532-6f01faa57fb0 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:53.437361 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252e467f-3ec2-4b4b-9532-6f01faa57fb0 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:53.448768 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application [None req-252e467f-3ec2-4b4b-9532-6f01faa57fb0 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 18 04:59:53.448768 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:53.448768 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 18 04:59:53.448768 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 18 04:59:53.448768 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 18 04:59:53.448768 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 18 04:59:53.448768 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 18 04:59:53.448768 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 18 04:59:53.448768 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 18 04:59:53.448768 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 04:59:53.448768 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 18 04:59:53.448768 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 04:59:53.448768 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:53.448768 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:59:53.448768 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:59:53.448768 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:59:53.448768 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:59:53.450039 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:59:53.450039 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:59:53.450039 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:59:53.450039 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:59:53.450039 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:59:53.450039 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:59:53.450039 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 18 04:59:53.450039 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 18 04:59:53.450039 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 18 04:59:53.450039 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 18 04:59:53.450039 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:59:53.450039 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:59:53.450039 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:59:53.450039 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:59:53.450039 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:59:53.450039 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:59:53.450886 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:59:53.450886 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application with Lock( Jun 18 04:59:53.450886 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:59:53.450886 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:59:53.450886 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:59:53.450886 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:59:53.450886 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:59:53.450886 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:59:53.450886 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:59:53.450886 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:59:53.450886 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 18 04:59:53.450886 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 18 04:59:53.450886 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 18 04:59:53.450886 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 04:59:53.450886 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 505/1026] 10.222.0.23 () {56 vars in 1105 bytes} [Sat Jun 18 04:59:53 2022] GET /identity/v3/services/image => generated 86 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 18 04:59:53.456073 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252e467f-3ec2-4b4b-9532-6f01faa57fb0 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:53.477609 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 522/1027] 10.222.0.23 () {60 vars in 1132 bytes} [Sat Jun 18 04:59:53 2022] POST /identity/v3/users => generated 285 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 18 04:59:53.571936 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252e467f-3ec2-4b4b-9532-6f01faa57fb0 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:53.610581 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d5a2a8-073b-4d37-b717-a53c74b19c29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:53.611274 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d5a2a8-073b-4d37-b717-a53c74b19c29 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:53.611547 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d5a2a8-073b-4d37-b717-a53c74b19c29 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:53.611827 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d5a2a8-073b-4d37-b717-a53c74b19c29 None admin] PATH_INFO: `/v3/services` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:53.612545 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d5a2a8-073b-4d37-b717-a53c74b19c29 None admin] RBAC: Adding query filter params (name=image) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:53.612861 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d5a2a8-073b-4d37-b717-a53c74b19c29 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:53.614370 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d5a2a8-073b-4d37-b717-a53c74b19c29 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:53.621116 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 506/1028] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Jun 18 04:59:53 2022] GET /identity/v3/services?name=image => generated 122 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 18 04:59:53.626999 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38f9d2e-52fd-4d23-bd7a-fc427751de54 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:53.737437 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38f9d2e-52fd-4d23-bd7a-fc427751de54 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:53.751042 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:53.772519 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6684a0d-b143-4ae6-9441-88467a1a94e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:53.773185 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6684a0d-b143-4ae6-9441-88467a1a94e5 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:53.773498 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6684a0d-b143-4ae6-9441-88467a1a94e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:53.773776 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6684a0d-b143-4ae6-9441-88467a1a94e5 None admin] PATH_INFO: `/v3/services` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:53.774475 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6684a0d-b143-4ae6-9441-88467a1a94e5 None admin] RBAC: Adding query filter params (type=image) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:53.774786 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6684a0d-b143-4ae6-9441-88467a1a94e5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:53.776271 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6684a0d-b143-4ae6-9441-88467a1a94e5 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:53.781696 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 523/1029] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Jun 18 04:59:53 2022] GET /identity/v3/services?type=image => generated 350 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 18 04:59:53.785218 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f443e2-64fa-43fc-8192-3f8dbfb18643 None None] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:53.785536 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f443e2-64fa-43fc-8192-3f8dbfb18643 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:53.785815 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f443e2-64fa-43fc-8192-3f8dbfb18643 None None] PATH_INFO: `/` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:53.788270 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6684a0d-b143-4ae6-9441-88467a1a94e5 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:53.788679 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 507/1030] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 04:59:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 04:59:53.793729 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da16656b-6a2f-4e7e-8422-ca9263cbe15b None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:53.794013 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da16656b-6a2f-4e7e-8422-ca9263cbe15b None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:53.797867 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da16656b-6a2f-4e7e-8422-ca9263cbe15b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:53.811564 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-da16656b-6a2f-4e7e-8422-ca9263cbe15b None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 04:59:53.812852 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-da16656b-6a2f-4e7e-8422-ca9263cbe15b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 04:59:53.823671 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:53.887397 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 508/1031] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 04:59:53 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 18 04:59:53.900825 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6684a0d-b143-4ae6-9441-88467a1a94e5 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:53.910593 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da16656b-6a2f-4e7e-8422-ca9263cbe15b None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:53.936856 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9ac133-4f3c-47ce-9fcc-c40cf6ad7eac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:53.937629 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9ac133-4f3c-47ce-9fcc-c40cf6ad7eac None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:53.937888 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9ac133-4f3c-47ce-9fcc-c40cf6ad7eac None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:53.938166 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9ac133-4f3c-47ce-9fcc-c40cf6ad7eac None admin] PATH_INFO: `/v3/endpoints` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:53.938955 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9ac133-4f3c-47ce-9fcc-c40cf6ad7eac None admin] RBAC: Adding query filter params (interface=public, service_id=11f3fe45377f4e128bdca480b6d0e8c6, region_id=RegionOne) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:53.939345 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9ac133-4f3c-47ce-9fcc-c40cf6ad7eac None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:53.940685 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9ac133-4f3c-47ce-9fcc-c40cf6ad7eac None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:53.947355 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 524/1032] 10.222.0.23 () {56 vars in 1251 bytes} [Sat Jun 18 04:59:53 2022] GET /identity/v3/endpoints?service_id=11f3fe45377f4e128bdca480b6d0e8c6&interface=public®ion_id=RegionOne => generated 194 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:53.952703 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9ac133-4f3c-47ce-9fcc-c40cf6ad7eac None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:54.016028 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da16656b-6a2f-4e7e-8422-ca9263cbe15b None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:54.061990 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:54.089838 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d779ee3f-a480-42bd-8f9f-c11b4ca249a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:54.090656 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d779ee3f-a480-42bd-8f9f-c11b4ca249a8 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:54.090940 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d779ee3f-a480-42bd-8f9f-c11b4ca249a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:54.091218 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d779ee3f-a480-42bd-8f9f-c11b4ca249a8 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:54.104757 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d779ee3f-a480-42bd-8f9f-c11b4ca249a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 18 04:59:54.118205 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application [None req-d779ee3f-a480-42bd-8f9f-c11b4ca249a8 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 18 04:59:54.118205 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:54.118205 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:59:54.118205 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:59:54.118205 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:59:54.118205 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:59:54.118205 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:59:54.118205 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:59:54.118205 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:59:54.118205 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:59:54.118205 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:59:54.118205 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:59:54.118205 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 18 04:59:54.118205 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 18 04:59:54.118205 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 18 04:59:54.118205 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 18 04:59:54.119131 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 18 04:59:54.119131 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 18 04:59:54.119131 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 18 04:59:54.119131 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 18 04:59:54.119131 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:59:54.119131 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:59:54.119131 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:59:54.119131 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:59:54.119131 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:59:54.119131 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:59:54.119131 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:59:54.119131 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application with Lock( Jun 18 04:59:54.119131 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:59:54.119131 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:59:54.119131 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:59:54.119131 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:59:54.119957 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:59:54.119957 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:59:54.119957 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:59:54.119957 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:59:54.119957 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 18 04:59:54.119957 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 18 04:59:54.119957 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 18 04:59:54.119957 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 18 04:59:54.119957 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 18 04:59:54.119957 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 18 04:59:54.119957 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 18 04:59:54.119957 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 04:59:54.120589 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 509/1033] 10.222.0.23 () {56 vars in 1120 bytes} [Sat Jun 18 04:59:53 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 18 04:59:54.129449 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d779ee3f-a480-42bd-8f9f-c11b4ca249a8 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:54.129601 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9ac133-4f3c-47ce-9fcc-c40cf6ad7eac None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:54.131441 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:54.163394 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825ff202-96fa-45f0-af93-ba6380fd8d98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:54.164060 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825ff202-96fa-45f0-af93-ba6380fd8d98 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:54.164347 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825ff202-96fa-45f0-af93-ba6380fd8d98 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:54.164635 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825ff202-96fa-45f0-af93-ba6380fd8d98 None admin] PATH_INFO: `/v3/services` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:54.165317 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825ff202-96fa-45f0-af93-ba6380fd8d98 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:54.166816 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825ff202-96fa-45f0-af93-ba6380fd8d98 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:54.173941 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 525/1034] 10.222.0.23 () {56 vars in 1087 bytes} [Sat Jun 18 04:59:53 2022] GET /identity/v3/services => generated 1957 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 18 04:59:54.245346 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d779ee3f-a480-42bd-8f9f-c11b4ca249a8 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:54.282968 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e09d6c-e6c1-484e-abef-b35f67bc95fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:54.283562 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e09d6c-e6c1-484e-abef-b35f67bc95fb None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:54.283844 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e09d6c-e6c1-484e-abef-b35f67bc95fb None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:54.284117 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e09d6c-e6c1-484e-abef-b35f67bc95fb None admin] PATH_INFO: `/v3/roles` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:54.284875 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e09d6c-e6c1-484e-abef-b35f67bc95fb None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:54.285188 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e09d6c-e6c1-484e-abef-b35f67bc95fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:54.287254 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e09d6c-e6c1-484e-abef-b35f67bc95fb None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:54.301440 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 510/1035] 10.222.0.23 () {56 vars in 1115 bytes} [Sat Jun 18 04:59:54 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:54.303446 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825ff202-96fa-45f0-af93-ba6380fd8d98 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:54.415335 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825ff202-96fa-45f0-af93-ba6380fd8d98 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:54.418536 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:54.449895 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883e553e-91ce-4362-9de2-e54c20b93eff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:54.450688 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883e553e-91ce-4362-9de2-e54c20b93eff None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:54.450974 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883e553e-91ce-4362-9de2-e54c20b93eff None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:54.451250 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883e553e-91ce-4362-9de2-e54c20b93eff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:54.465690 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883e553e-91ce-4362-9de2-e54c20b93eff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:54.467150 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883e553e-91ce-4362-9de2-e54c20b93eff None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:54.484098 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-883e553e-91ce-4362-9de2-e54c20b93eff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 04:59:54.484098 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:54.484098 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:59:54.484098 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:59:54.484098 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:59:54.484098 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:59:54.484098 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:59:54.484098 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:59:54.484098 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:59:54.484098 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:59:54.484098 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:59:54.484098 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:59:54.484098 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 18 04:59:54.484098 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 18 04:59:54.484098 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 18 04:59:54.484985 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 18 04:59:54.484985 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:59:54.484985 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:59:54.484985 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:59:54.484985 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:59:54.484985 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:59:54.484985 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:59:54.484985 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:59:54.484985 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jun 18 04:59:54.484985 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:59:54.484985 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:59:54.484985 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:59:54.484985 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:59:54.484985 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:59:54.484985 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:59:54.484985 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:59:54.484985 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:59:54.485895 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 18 04:59:54.485895 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 18 04:59:54.485895 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 04:59:54.485895 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 04:59:54.485895 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 526/1036] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Jun 18 04:59:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:59:54.491027 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e09d6c-e6c1-484e-abef-b35f67bc95fb None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:54.559256 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:54.594696 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e09d6c-e6c1-484e-abef-b35f67bc95fb None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:54.629859 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81775d6-0b27-4f92-a84c-fad80e5b5ed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:54.631463 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81775d6-0b27-4f92-a84c-fad80e5b5ed4 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:54.631463 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81775d6-0b27-4f92-a84c-fad80e5b5ed4 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:54.631814 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81775d6-0b27-4f92-a84c-fad80e5b5ed4 None admin] PATH_INFO: `/v3/domains` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:54.632505 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81775d6-0b27-4f92-a84c-fad80e5b5ed4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:54.632815 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81775d6-0b27-4f92-a84c-fad80e5b5ed4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:54.634204 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81775d6-0b27-4f92-a84c-fad80e5b5ed4 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:54.647790 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 511/1037] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Jun 18 04:59:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:54.653296 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883e553e-91ce-4362-9de2-e54c20b93eff None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:54.756618 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883e553e-91ce-4362-9de2-e54c20b93eff None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:54.789550 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbd7956-5595-488c-a86b-e60605a8bc5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:54.790331 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbd7956-5595-488c-a86b-e60605a8bc5f None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:54.790615 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbd7956-5595-488c-a86b-e60605a8bc5f None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:54.790886 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbd7956-5595-488c-a86b-e60605a8bc5f None admin] PATH_INFO: `/v3/users/nova` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:54.798388 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbd7956-5595-488c-a86b-e60605a8bc5f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:54.799830 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbd7956-5595-488c-a86b-e60605a8bc5f None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:54.814133 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-cfbd7956-5595-488c-a86b-e60605a8bc5f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 18 04:59:54.814133 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:54.814133 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:59:54.814133 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:59:54.814133 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:59:54.814133 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:59:54.814133 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:59:54.814133 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:59:54.814133 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:59:54.814133 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:59:54.814133 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:59:54.814133 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:59:54.814133 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 18 04:59:54.814133 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 18 04:59:54.814133 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 18 04:59:54.814133 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 18 04:59:54.815094 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:59:54.815094 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:59:54.815094 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 18 04:59:54.815094 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 18 04:59:54.815094 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 18 04:59:54.815094 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 18 04:59:54.815094 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:59:54.815094 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:59:54.815094 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:59:54.815094 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:59:54.815094 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:59:54.815094 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jun 18 04:59:54.815094 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:59:54.815094 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:59:54.815094 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:59:54.815094 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:59:54.815094 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:59:54.815094 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:59:54.816082 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:59:54.816082 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:59:54.816082 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 18 04:59:54.816082 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 18 04:59:54.816082 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 18 04:59:54.816082 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 18 04:59:54.816082 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 18 04:59:54.816082 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 18 04:59:54.816082 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 18 04:59:54.816082 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 04:59:54.817719 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 527/1038] 10.222.0.23 () {56 vars in 1093 bytes} [Sat Jun 18 04:59:54 2022] GET /identity/v3/users/nova => generated 82 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:59:54.821053 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81775d6-0b27-4f92-a84c-fad80e5b5ed4 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:54.921915 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81775d6-0b27-4f92-a84c-fad80e5b5ed4 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:54.947383 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:54.956336 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62a2504-4ce8-43fe-98c1-af85eec42e9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:54.956998 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62a2504-4ce8-43fe-98c1-af85eec42e9e None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:54.957281 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62a2504-4ce8-43fe-98c1-af85eec42e9e None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:54.958974 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62a2504-4ce8-43fe-98c1-af85eec42e9e None admin] PATH_INFO: `/v3/users` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:54.959069 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62a2504-4ce8-43fe-98c1-af85eec42e9e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:54.959599 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62a2504-4ce8-43fe-98c1-af85eec42e9e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:54.960813 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62a2504-4ce8-43fe-98c1-af85eec42e9e None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:54.981544 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 512/1039] 10.222.0.23 () {56 vars in 1133 bytes} [Sat Jun 18 04:59:54 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:54.984067 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbd7956-5595-488c-a86b-e60605a8bc5f None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:55.003271 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:55.090299 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbd7956-5595-488c-a86b-e60605a8bc5f None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:55.127278 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d1c400-3cec-478f-832d-b41889b64f77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:55.127919 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d1c400-3cec-478f-832d-b41889b64f77 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:55.128201 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d1c400-3cec-478f-832d-b41889b64f77 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:55.128474 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d1c400-3cec-478f-832d-b41889b64f77 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:55.198704 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d1c400-3cec-478f-832d-b41889b64f77 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:55.200163 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d1c400-3cec-478f-832d-b41889b64f77 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:55.217741 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:55.309625 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 528/1040] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Jun 18 04:59:54 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 18 04:59:55.310849 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62a2504-4ce8-43fe-98c1-af85eec42e9e None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:55.421528 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62a2504-4ce8-43fe-98c1-af85eec42e9e None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:55.457120 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc261e84-11c0-4f10-8c70-b695265f89c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:55.457946 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc261e84-11c0-4f10-8c70-b695265f89c8 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:55.458229 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc261e84-11c0-4f10-8c70-b695265f89c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:55.458567 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc261e84-11c0-4f10-8c70-b695265f89c8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:55.469047 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc261e84-11c0-4f10-8c70-b695265f89c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:55.470515 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc261e84-11c0-4f10-8c70-b695265f89c8 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:55.488111 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application [None req-bc261e84-11c0-4f10-8c70-b695265f89c8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 04:59:55.488111 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:55.488111 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:59:55.488111 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:59:55.488111 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:59:55.488111 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:59:55.488111 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:59:55.488111 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:59:55.488111 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:59:55.488111 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:59:55.488111 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:59:55.488111 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:59:55.488111 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 18 04:59:55.488111 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 18 04:59:55.488111 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 18 04:59:55.489034 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 18 04:59:55.489034 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:59:55.489034 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:59:55.489034 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:59:55.489034 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:59:55.489034 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:59:55.489034 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:59:55.489034 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:59:55.489034 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application with Lock( Jun 18 04:59:55.489034 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:59:55.489034 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:59:55.489034 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:59:55.489034 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:59:55.489034 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:59:55.489034 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:59:55.489034 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:59:55.489034 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:59:55.491326 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 18 04:59:55.491326 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 18 04:59:55.491326 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 04:59:55.491326 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 04:59:55.493735 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 513/1041] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Jun 18 04:59:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:59:55.495721 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d1c400-3cec-478f-832d-b41889b64f77 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:55.596588 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a646b0c0-337e-476e-b9cb-7f73e4777389 None None] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:55.596906 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a646b0c0-337e-476e-b9cb-7f73e4777389 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:55.597117 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d1c400-3cec-478f-832d-b41889b64f77 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:55.597224 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a646b0c0-337e-476e-b9cb-7f73e4777389 None None] PATH_INFO: `/` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:55.598705 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 514/1042] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 04:59:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 18 04:59:55.603764 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c4c771-f478-489e-bf42-9817527daec4 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:55.604059 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c4c771-f478-489e-bf42-9817527daec4 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:55.604337 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c4c771-f478-489e-bf42-9817527daec4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:55.614399 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:55.621738 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-55c4c771-f478-489e-bf42-9817527daec4 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 04:59:55.622938 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-55c4c771-f478-489e-bf42-9817527daec4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 04:59:55.634319 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132535d6-57f0-4388-9697-870d9d52ed7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:55.644801 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132535d6-57f0-4388-9697-870d9d52ed7a None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:55.644801 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132535d6-57f0-4388-9697-870d9d52ed7a None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:55.644801 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132535d6-57f0-4388-9697-870d9d52ed7a None admin] PATH_INFO: `/v3/domains` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:55.644801 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132535d6-57f0-4388-9697-870d9d52ed7a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:55.644801 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132535d6-57f0-4388-9697-870d9d52ed7a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:55.644801 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132535d6-57f0-4388-9697-870d9d52ed7a None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:55.654078 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 529/1043] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Jun 18 04:59:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 18 04:59:55.655083 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132535d6-57f0-4388-9697-870d9d52ed7a None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:55.658816 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:55.713751 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 515/1044] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 04:59:55 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 18 04:59:55.729224 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c4c771-f478-489e-bf42-9817527daec4 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:55.761966 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132535d6-57f0-4388-9697-870d9d52ed7a None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:55.798632 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ecf1dc-eb9e-4b34-9edf-299d87bccccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:55.799421 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ecf1dc-eb9e-4b34-9edf-299d87bccccf None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:55.800032 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ecf1dc-eb9e-4b34-9edf-299d87bccccf None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:55.800032 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ecf1dc-eb9e-4b34-9edf-299d87bccccf None admin] PATH_INFO: `/v3/projects/service` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:55.807626 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ecf1dc-eb9e-4b34-9edf-299d87bccccf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:55.809073 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ecf1dc-eb9e-4b34-9edf-299d87bccccf None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:55.824504 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-b4ecf1dc-eb9e-4b34-9edf-299d87bccccf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 18 04:59:55.824504 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:55.824504 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:59:55.824504 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:59:55.824504 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:59:55.824504 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:59:55.824504 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:59:55.824504 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:59:55.824504 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:59:55.824504 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:59:55.824504 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:59:55.824504 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:59:55.824504 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 18 04:59:55.824504 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 18 04:59:55.824504 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 18 04:59:55.825646 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 18 04:59:55.825646 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:59:55.825646 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:59:55.825646 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:59:55.825646 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:59:55.825646 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:59:55.825646 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:59:55.825646 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:59:55.825646 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jun 18 04:59:55.825646 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:59:55.825646 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:59:55.825646 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:59:55.825646 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:59:55.825646 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:59:55.825646 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:59:55.825646 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:59:55.825646 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:59:55.826947 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 18 04:59:55.826947 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 18 04:59:55.826947 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 18 04:59:55.826947 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 18 04:59:55.826947 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 18 04:59:55.826947 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 18 04:59:55.826947 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 18 04:59:55.826947 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 04:59:55.826947 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 530/1045] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Jun 18 04:59:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:59:55.831252 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ecf1dc-eb9e-4b34-9edf-299d87bccccf None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:55.839332 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c4c771-f478-489e-bf42-9817527daec4 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:55.868645 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:55.875178 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7475ce5-4684-4b87-a5f2-ee34c1aedc68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:55.875968 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7475ce5-4684-4b87-a5f2-ee34c1aedc68 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:55.876256 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7475ce5-4684-4b87-a5f2-ee34c1aedc68 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:55.876594 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7475ce5-4684-4b87-a5f2-ee34c1aedc68 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:55.888322 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7475ce5-4684-4b87-a5f2-ee34c1aedc68 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 18 04:59:55.895938 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application [None req-a7475ce5-4684-4b87-a5f2-ee34c1aedc68 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 18 04:59:55.895938 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:55.895938 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:59:55.895938 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:59:55.895938 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:59:55.895938 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:59:55.895938 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:59:55.895938 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:59:55.895938 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:59:55.895938 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:59:55.895938 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:59:55.895938 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:59:55.895938 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 18 04:59:55.895938 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 18 04:59:55.895938 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 18 04:59:55.895938 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 18 04:59:55.896880 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 18 04:59:55.896880 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 18 04:59:55.896880 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 18 04:59:55.896880 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 18 04:59:55.896880 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:59:55.896880 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:59:55.896880 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:59:55.896880 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:59:55.896880 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:59:55.896880 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:59:55.896880 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:59:55.896880 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application with Lock( Jun 18 04:59:55.896880 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:59:55.896880 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:59:55.896880 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:59:55.896880 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:59:55.897768 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:59:55.897768 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:59:55.897768 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:59:55.897768 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:59:55.897768 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 18 04:59:55.897768 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 18 04:59:55.897768 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 18 04:59:55.897768 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 18 04:59:55.897768 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 18 04:59:55.897768 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 18 04:59:55.897768 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 18 04:59:55.897768 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 04:59:55.899171 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 516/1046] 10.222.0.23 () {56 vars in 1096 bytes} [Sat Jun 18 04:59:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 18 04:59:55.904807 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7475ce5-4684-4b87-a5f2-ee34c1aedc68 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:55.936798 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:55.941337 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ecf1dc-eb9e-4b34-9edf-299d87bccccf None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:55.977924 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756d0593-9be0-4e76-8104-bd4ee98bb2b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:55.978579 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756d0593-9be0-4e76-8104-bd4ee98bb2b3 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:55.978869 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756d0593-9be0-4e76-8104-bd4ee98bb2b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:55.979145 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756d0593-9be0-4e76-8104-bd4ee98bb2b3 None admin] PATH_INFO: `/v3/projects` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:55.979932 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756d0593-9be0-4e76-8104-bd4ee98bb2b3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:55.980245 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756d0593-9be0-4e76-8104-bd4ee98bb2b3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:55.981711 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756d0593-9be0-4e76-8104-bd4ee98bb2b3 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:55.993845 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 531/1047] 10.222.0.23 () {56 vars in 1148 bytes} [Sat Jun 18 04:59:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 18 04:59:55.999286 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756d0593-9be0-4e76-8104-bd4ee98bb2b3 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:56.022240 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7475ce5-4684-4b87-a5f2-ee34c1aedc68 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:56.058297 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f393d4-1844-472a-b518-5498c41bde1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:56.059009 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f393d4-1844-472a-b518-5498c41bde1a None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:56.059293 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f393d4-1844-472a-b518-5498c41bde1a None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:56.059621 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f393d4-1844-472a-b518-5498c41bde1a None admin] PATH_INFO: `/v3/roles` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:56.060362 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f393d4-1844-472a-b518-5498c41bde1a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:56.060671 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f393d4-1844-472a-b518-5498c41bde1a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:56.062130 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f393d4-1844-472a-b518-5498c41bde1a None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:56.071023 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 517/1048] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Jun 18 04:59:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:56.076241 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f393d4-1844-472a-b518-5498c41bde1a None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:56.103288 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:56.110616 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756d0593-9be0-4e76-8104-bd4ee98bb2b3 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:56.147031 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe3e5f6-c21a-4d95-b826-953422979485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:56.147643 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe3e5f6-c21a-4d95-b826-953422979485 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:56.147925 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe3e5f6-c21a-4d95-b826-953422979485 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:56.148201 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe3e5f6-c21a-4d95-b826-953422979485 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:56.149020 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe3e5f6-c21a-4d95-b826-953422979485 None admin] RBAC: Adding query filter params (role.id=2495da35026945aabd29678096996e70, scope.project.id=c95f8f3084c5472cb8364b7c57207394, user.id=15e7f45ee0744c1d9602187eac8e8b80) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:56.149338 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe3e5f6-c21a-4d95-b826-953422979485 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:56.150810 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe3e5f6-c21a-4d95-b826-953422979485 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:56.156468 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 532/1049] 10.222.0.23 () {56 vars in 1374 bytes} [Sat Jun 18 04:59:55 2022] GET /identity/v3/role_assignments?user.id=15e7f45ee0744c1d9602187eac8e8b80&scope.project.id=c95f8f3084c5472cb8364b7c57207394&role.id=2495da35026945aabd29678096996e70 => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:56.183354 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:56.197633 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f393d4-1844-472a-b518-5498c41bde1a None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:56.244878 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446abb14-ecf2-4ca6-8af7-131cbdafc833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:56.245804 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446abb14-ecf2-4ca6-8af7-131cbdafc833 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:56.246093 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446abb14-ecf2-4ca6-8af7-131cbdafc833 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:56.246370 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446abb14-ecf2-4ca6-8af7-131cbdafc833 None admin] PATH_INFO: `/v3/users/25348480076647289315bab6a97269a0` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:56.261839 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446abb14-ecf2-4ca6-8af7-131cbdafc833 None admin] RBAC: Authorizing `identity:get_user(user_id=25348480076647289315bab6a97269a0)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:56.263320 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446abb14-ecf2-4ca6-8af7-131cbdafc833 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:56.269415 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 518/1050] 10.222.0.23 () {56 vars in 1177 bytes} [Sat Jun 18 04:59:56 2022] GET /identity/v3/users/25348480076647289315bab6a97269a0 => generated 285 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 18 04:59:56.269604 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381822dc-0d89-4750-8460-2d0a85fda882 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:56.269604 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381822dc-0d89-4750-8460-2d0a85fda882 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:56.269604 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381822dc-0d89-4750-8460-2d0a85fda882 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:56.269604 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 533/1051] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 04:59:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 18 04:59:56.272586 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446abb14-ecf2-4ca6-8af7-131cbdafc833 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:56.274597 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ea9ffb-0480-4a0f-a2fb-c6ad00039381 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:56.274881 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ea9ffb-0480-4a0f-a2fb-c6ad00039381 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:56.275224 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ea9ffb-0480-4a0f-a2fb-c6ad00039381 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:56.293961 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-29ea9ffb-0480-4a0f-a2fb-c6ad00039381 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 04:59:56.295186 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-29ea9ffb-0480-4a0f-a2fb-c6ad00039381 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 04:59:56.372204 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 534/1052] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 04:59:56 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 18 04:59:56.392454 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446abb14-ecf2-4ca6-8af7-131cbdafc833 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:56.394728 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ea9ffb-0480-4a0f-a2fb-c6ad00039381 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:56.431436 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb79c4b-dafd-4037-bbe2-6826f64d822c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:56.432132 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb79c4b-dafd-4037-bbe2-6826f64d822c None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:56.432417 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb79c4b-dafd-4037-bbe2-6826f64d822c None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:56.432697 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb79c4b-dafd-4037-bbe2-6826f64d822c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:56.449338 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:56.496718 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:56.503153 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb79c4b-dafd-4037-bbe2-6826f64d822c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:56.504627 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb79c4b-dafd-4037-bbe2-6826f64d822c None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:56.507683 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ea9ffb-0480-4a0f-a2fb-c6ad00039381 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:56.544664 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896f6946-8b75-46e8-b9fe-7e237afa632f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:56.545481 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896f6946-8b75-46e8-b9fe-7e237afa632f None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:56.545763 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896f6946-8b75-46e8-b9fe-7e237afa632f None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:56.546038 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896f6946-8b75-46e8-b9fe-7e237afa632f None admin] PATH_INFO: `/v3/services/image` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:56.546687 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896f6946-8b75-46e8-b9fe-7e237afa632f None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:56.548459 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896f6946-8b75-46e8-b9fe-7e237afa632f None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:56.561828 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-896f6946-8b75-46e8-b9fe-7e237afa632f None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 18 04:59:56.561828 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:56.561828 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 18 04:59:56.561828 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 18 04:59:56.561828 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 18 04:59:56.561828 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 18 04:59:56.561828 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 18 04:59:56.561828 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 18 04:59:56.561828 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 18 04:59:56.561828 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 04:59:56.561828 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 18 04:59:56.561828 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 04:59:56.561828 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:56.561828 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:59:56.561828 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:59:56.561828 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:59:56.561828 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:59:56.563040 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:59:56.563040 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:59:56.563040 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:59:56.563040 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:59:56.563040 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:59:56.563040 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:59:56.563040 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 18 04:59:56.563040 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 18 04:59:56.563040 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 18 04:59:56.563040 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 18 04:59:56.563040 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:59:56.563040 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:59:56.563040 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:59:56.563040 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:59:56.563040 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:59:56.563040 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:59:56.563997 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:59:56.563997 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jun 18 04:59:56.563997 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:59:56.563997 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:59:56.563997 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:59:56.563997 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:59:56.563997 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:59:56.563997 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:59:56.563997 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:59:56.563997 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:59:56.563997 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 18 04:59:56.563997 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 18 04:59:56.563997 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 18 04:59:56.563997 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 04:59:56.563997 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 535/1053] 10.222.0.23 () {56 vars in 1105 bytes} [Sat Jun 18 04:59:56 2022] GET /identity/v3/services/image => generated 86 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 18 04:59:56.569535 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896f6946-8b75-46e8-b9fe-7e237afa632f None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:56.623837 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 519/1054] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Jun 18 04:59:56 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 18 04:59:56.624628 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb79c4b-dafd-4037-bbe2-6826f64d822c None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:56.675629 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:56.679799 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896f6946-8b75-46e8-b9fe-7e237afa632f None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:56.720246 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75929c4f-cc72-4a03-8702-3e7c7583f8ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:56.720820 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75929c4f-cc72-4a03-8702-3e7c7583f8ad None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:56.721101 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75929c4f-cc72-4a03-8702-3e7c7583f8ad None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:56.721398 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75929c4f-cc72-4a03-8702-3e7c7583f8ad None admin] PATH_INFO: `/v3/services` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:56.722097 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75929c4f-cc72-4a03-8702-3e7c7583f8ad None admin] RBAC: Adding query filter params (name=image) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:56.722406 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75929c4f-cc72-4a03-8702-3e7c7583f8ad None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:56.723776 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75929c4f-cc72-4a03-8702-3e7c7583f8ad None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:56.728928 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 536/1055] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Jun 18 04:59:56 2022] GET /identity/v3/services?name=image => generated 122 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:56.734437 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75929c4f-cc72-4a03-8702-3e7c7583f8ad None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:56.737577 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb79c4b-dafd-4037-bbe2-6826f64d822c None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:56.753807 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:56.775284 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7f2144-5289-4f7f-9408-bbd8c0cc8966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:56.776104 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7f2144-5289-4f7f-9408-bbd8c0cc8966 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:56.776430 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7f2144-5289-4f7f-9408-bbd8c0cc8966 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:56.776703 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7f2144-5289-4f7f-9408-bbd8c0cc8966 None admin] PATH_INFO: `/v3/projects/8c469baa2dc5484abd09e54922200ffd` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:56.777922 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7f2144-5289-4f7f-9408-bbd8c0cc8966 None admin] RBAC: Authorizing `identity:get_project(project_id=8c469baa2dc5484abd09e54922200ffd)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:56.779340 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7f2144-5289-4f7f-9408-bbd8c0cc8966 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:56.785849 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 520/1056] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Jun 18 04:59:56 2022] GET /identity/v3/projects/8c469baa2dc5484abd09e54922200ffd => generated 311 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 18 04:59:56.787527 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7f2144-5289-4f7f-9408-bbd8c0cc8966 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:56.843433 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75929c4f-cc72-4a03-8702-3e7c7583f8ad None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:56.879753 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c7a9b3-cd6a-4357-9864-0cda2cb08852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:56.881336 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c7a9b3-cd6a-4357-9864-0cda2cb08852 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:56.881649 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c7a9b3-cd6a-4357-9864-0cda2cb08852 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:56.882090 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c7a9b3-cd6a-4357-9864-0cda2cb08852 None admin] PATH_INFO: `/v3/services` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:56.883161 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c7a9b3-cd6a-4357-9864-0cda2cb08852 None admin] RBAC: Adding query filter params (type=image) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:56.883480 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c7a9b3-cd6a-4357-9864-0cda2cb08852 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:56.885405 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c7a9b3-cd6a-4357-9864-0cda2cb08852 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:56.896410 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 537/1057] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Jun 18 04:59:56 2022] GET /identity/v3/services?type=image => generated 350 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:56.898171 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c7a9b3-cd6a-4357-9864-0cda2cb08852 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:56.900485 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7f2144-5289-4f7f-9408-bbd8c0cc8966 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:56.916478 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:56.937375 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96b2cd3-05a5-49b1-b1e3-a2cca14ea4aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:56.938045 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96b2cd3-05a5-49b1-b1e3-a2cca14ea4aa None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:56.938335 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96b2cd3-05a5-49b1-b1e3-a2cca14ea4aa None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:56.938611 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96b2cd3-05a5-49b1-b1e3-a2cca14ea4aa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:56.939542 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96b2cd3-05a5-49b1-b1e3-a2cca14ea4aa None admin] RBAC: Adding query filter params (role.id=21c65f2c51bd4dd283d13507d2714467, scope.project.id=8c469baa2dc5484abd09e54922200ffd, user.id=25348480076647289315bab6a97269a0) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:56.939860 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96b2cd3-05a5-49b1-b1e3-a2cca14ea4aa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:56.941552 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96b2cd3-05a5-49b1-b1e3-a2cca14ea4aa None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:56.948031 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 521/1058] 10.222.0.23 () {56 vars in 1374 bytes} [Sat Jun 18 04:59:56 2022] GET /identity/v3/role_assignments?user.id=25348480076647289315bab6a97269a0&scope.project.id=8c469baa2dc5484abd09e54922200ffd&role.id=21c65f2c51bd4dd283d13507d2714467 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:57.000981 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:57.020603 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c7a9b3-cd6a-4357-9864-0cda2cb08852 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:57.058532 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffa5f7b-43c4-4b48-8d37-cf6928ba7788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:57.059199 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffa5f7b-43c4-4b48-8d37-cf6928ba7788 None admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:57.059480 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffa5f7b-43c4-4b48-8d37-cf6928ba7788 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:57.059754 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffa5f7b-43c4-4b48-8d37-cf6928ba7788 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:57.060509 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffa5f7b-43c4-4b48-8d37-cf6928ba7788 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:57.061955 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffa5f7b-43c4-4b48-8d37-cf6928ba7788 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:57.172676 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 538/1059] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Jun 18 04:59:56 2022] POST /identity/v3/endpoints => generated 327 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 04:59:58.186931 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c48912b-aed1-4f23-8969-3a4a7656a22b None None] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:58.187445 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c48912b-aed1-4f23-8969-3a4a7656a22b None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:58.187558 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c48912b-aed1-4f23-8969-3a4a7656a22b None None] PATH_INFO: `/` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:58.188995 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 522/1060] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 04:59:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 04:59:58.194641 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014c669c-2bfe-4878-ad36-46019653cc20 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:58.195050 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014c669c-2bfe-4878-ad36-46019653cc20 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:58.195224 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014c669c-2bfe-4878-ad36-46019653cc20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:58.212691 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-014c669c-2bfe-4878-ad36-46019653cc20 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 04:59:58.213909 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-014c669c-2bfe-4878-ad36-46019653cc20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 04:59:58.293523 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 539/1061] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 04:59:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 18 04:59:58.311578 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c48912b-aed1-4f23-8969-3a4a7656a22b None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:58.425403 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c48912b-aed1-4f23-8969-3a4a7656a22b None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:58.460777 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff41273-6540-4207-8db2-bca9c389cc47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:58.461540 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff41273-6540-4207-8db2-bca9c389cc47 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:58.461828 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff41273-6540-4207-8db2-bca9c389cc47 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:58.462103 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff41273-6540-4207-8db2-bca9c389cc47 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:58.472191 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff41273-6540-4207-8db2-bca9c389cc47 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 18 04:59:58.480347 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application [None req-eff41273-6540-4207-8db2-bca9c389cc47 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 18 04:59:58.480347 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:58.480347 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:59:58.480347 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:59:58.480347 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:59:58.480347 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:59:58.480347 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:59:58.480347 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:59:58.480347 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:59:58.480347 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:59:58.480347 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:59:58.480347 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:59:58.480347 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 18 04:59:58.480347 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 18 04:59:58.480347 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 18 04:59:58.480347 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 18 04:59:58.481285 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 18 04:59:58.481285 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 18 04:59:58.481285 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 18 04:59:58.481285 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 18 04:59:58.481285 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:59:58.481285 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:59:58.481285 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:59:58.481285 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:59:58.481285 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:59:58.481285 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:59:58.481285 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:59:58.481285 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application with Lock( Jun 18 04:59:58.481285 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:59:58.481285 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:59:58.481285 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:59:58.481285 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:59:58.487547 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:59:58.487547 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:59:58.487547 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:59:58.487547 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:59:58.487547 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 18 04:59:58.487547 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 18 04:59:58.487547 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 18 04:59:58.487547 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 18 04:59:58.487547 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 18 04:59:58.487547 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 18 04:59:58.487547 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 18 04:59:58.487547 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 04:59:58.487547 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 523/1062] 10.222.0.23 () {56 vars in 1120 bytes} [Sat Jun 18 04:59:58 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:59:58.490164 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014c669c-2bfe-4878-ad36-46019653cc20 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:58.554677 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:58.592060 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014c669c-2bfe-4878-ad36-46019653cc20 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:58.627102 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4de431-be56-4565-bd8f-e8f4be261bf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:58.627771 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4de431-be56-4565-bd8f-e8f4be261bf7 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:58.628052 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4de431-be56-4565-bd8f-e8f4be261bf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:58.628328 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4de431-be56-4565-bd8f-e8f4be261bf7 None admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:58.629066 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4de431-be56-4565-bd8f-e8f4be261bf7 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:58.629376 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4de431-be56-4565-bd8f-e8f4be261bf7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:58.631453 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4de431-be56-4565-bd8f-e8f4be261bf7 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:58.645092 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 540/1063] 10.222.0.23 () {56 vars in 1115 bytes} [Sat Jun 18 04:59:58 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 18 04:59:58.646425 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff41273-6540-4207-8db2-bca9c389cc47 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:58.688543 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:58.752013 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff41273-6540-4207-8db2-bca9c389cc47 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:58.785972 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9541c245-4998-4f21-bc52-0acd90a83443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:58.786820 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9541c245-4998-4f21-bc52-0acd90a83443 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:58.787109 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9541c245-4998-4f21-bc52-0acd90a83443 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:58.787423 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9541c245-4998-4f21-bc52-0acd90a83443 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:58.798491 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9541c245-4998-4f21-bc52-0acd90a83443 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:58.799934 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9541c245-4998-4f21-bc52-0acd90a83443 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:58.818365 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application [None req-9541c245-4998-4f21-bc52-0acd90a83443 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 04:59:58.818365 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:58.818365 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:59:58.818365 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:59:58.818365 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:59:58.818365 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:59:58.818365 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:59:58.818365 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:59:58.818365 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:59:58.818365 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:59:58.818365 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:59:58.818365 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:59:58.818365 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 18 04:59:58.818365 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 18 04:59:58.818365 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 18 04:59:58.819283 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 18 04:59:58.819283 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:59:58.819283 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:59:58.819283 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:59:58.819283 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:59:58.819283 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:59:58.819283 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:59:58.819283 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:59:58.819283 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application with Lock( Jun 18 04:59:58.819283 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:59:58.819283 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:59:58.819283 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:59:58.819283 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:59:58.819283 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:59:58.819283 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:59:58.819283 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:59:58.819283 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:59:58.820199 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 18 04:59:58.820199 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 18 04:59:58.820199 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 04:59:58.820199 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 04:59:58.821711 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 524/1064] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Jun 18 04:59:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:59:58.825318 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4de431-be56-4565-bd8f-e8f4be261bf7 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:58.928876 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4de431-be56-4565-bd8f-e8f4be261bf7 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:58.954718 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:58.963912 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc66886-7eb3-4eac-ba0e-d0dbd01ef41e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:58.967599 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc66886-7eb3-4eac-ba0e-d0dbd01ef41e None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:58.967599 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc66886-7eb3-4eac-ba0e-d0dbd01ef41e None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:58.967599 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc66886-7eb3-4eac-ba0e-d0dbd01ef41e None admin] PATH_INFO: `/v3/domains` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:58.967599 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc66886-7eb3-4eac-ba0e-d0dbd01ef41e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:58.967599 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc66886-7eb3-4eac-ba0e-d0dbd01ef41e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:58.967599 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc66886-7eb3-4eac-ba0e-d0dbd01ef41e None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:58.980001 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 541/1065] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Jun 18 04:59:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:58.985430 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9541c245-4998-4f21-bc52-0acd90a83443 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:59.040069 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77268f80-0c89-4828-ba80-3a430c7d1e5d None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:59.040267 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77268f80-0c89-4828-ba80-3a430c7d1e5d None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:59.040532 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77268f80-0c89-4828-ba80-3a430c7d1e5d None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:59.042065 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 542/1066] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 04:59:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 18 04:59:59.046817 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c07580-b357-4a2a-8ef6-19b425440912 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:59.047104 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c07580-b357-4a2a-8ef6-19b425440912 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:59.047381 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c07580-b357-4a2a-8ef6-19b425440912 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:59.066026 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-b1c07580-b357-4a2a-8ef6-19b425440912 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 04:59:59.067221 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-b1c07580-b357-4a2a-8ef6-19b425440912 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 04:59:59.081678 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:59.092698 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9541c245-4998-4f21-bc52-0acd90a83443 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:59.130333 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424483ea-d679-4eaa-b1e0-569fe6a61489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:59.131254 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424483ea-d679-4eaa-b1e0-569fe6a61489 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:59.131541 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424483ea-d679-4eaa-b1e0-569fe6a61489 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:59.131818 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424483ea-d679-4eaa-b1e0-569fe6a61489 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:59.139509 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 543/1067] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 04:59:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 18 04:59:59.140842 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424483ea-d679-4eaa-b1e0-569fe6a61489 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:59.142326 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424483ea-d679-4eaa-b1e0-569fe6a61489 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:59.158412 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application [None req-424483ea-d679-4eaa-b1e0-569fe6a61489 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 18 04:59:59.158412 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:59.158412 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:59:59.158412 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:59:59.158412 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:59:59.158412 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:59:59.158412 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:59:59.158412 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:59:59.158412 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:59:59.158412 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:59:59.158412 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:59:59.158412 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:59:59.158412 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 18 04:59:59.158412 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 18 04:59:59.158412 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 18 04:59:59.158412 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 18 04:59:59.159408 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:59:59.159408 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:59:59.159408 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 18 04:59:59.159408 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 18 04:59:59.159408 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 18 04:59:59.159408 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 18 04:59:59.159408 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:59:59.159408 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:59:59.159408 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:59:59.159408 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:59:59.159408 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:59:59.159408 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application with Lock( Jun 18 04:59:59.159408 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:59:59.159408 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:59:59.159408 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:59:59.159408 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:59:59.159408 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:59:59.159408 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:59:59.160376 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:59:59.160376 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:59:59.160376 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 18 04:59:59.160376 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 18 04:59:59.160376 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 18 04:59:59.160376 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 18 04:59:59.160376 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 18 04:59:59.160376 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 18 04:59:59.160376 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 18 04:59:59.160376 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 04:59:59.163307 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c07580-b357-4a2a-8ef6-19b425440912 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:59.164739 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 525/1068] 10.222.0.23 () {56 vars in 1093 bytes} [Sat Jun 18 04:59:58 2022] GET /identity/v3/users/nova => generated 82 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:59:59.165610 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424483ea-d679-4eaa-b1e0-569fe6a61489 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:59.268643 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424483ea-d679-4eaa-b1e0-569fe6a61489 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:59.270278 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c07580-b357-4a2a-8ef6-19b425440912 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:59.287249 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:59.303401 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e07d293-f510-4970-98ea-0afa22820db9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:59.303998 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e07d293-f510-4970-98ea-0afa22820db9 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:59.304284 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e07d293-f510-4970-98ea-0afa22820db9 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:59.304560 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e07d293-f510-4970-98ea-0afa22820db9 None admin] PATH_INFO: `/v3/users` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:59.305409 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e07d293-f510-4970-98ea-0afa22820db9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:59.305656 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd01e35-4779-43c6-a98b-b2f5e96a923a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:59.305784 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e07d293-f510-4970-98ea-0afa22820db9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:59.306442 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd01e35-4779-43c6-a98b-b2f5e96a923a None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:59.306727 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd01e35-4779-43c6-a98b-b2f5e96a923a None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:59.307004 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd01e35-4779-43c6-a98b-b2f5e96a923a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:59.307170 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e07d293-f510-4970-98ea-0afa22820db9 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:59.316846 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd01e35-4779-43c6-a98b-b2f5e96a923a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 18 04:59:59.325278 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-6cd01e35-4779-43c6-a98b-b2f5e96a923a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 18 04:59:59.325278 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:59.325278 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:59:59.325278 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:59:59.325278 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:59:59.325278 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:59:59.325278 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:59:59.325278 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:59:59.325278 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:59:59.325278 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:59:59.325278 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:59:59.325278 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:59:59.325278 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 18 04:59:59.325278 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 18 04:59:59.325278 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 18 04:59:59.325278 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 18 04:59:59.326225 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 18 04:59:59.326225 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 18 04:59:59.326225 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 18 04:59:59.326225 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 18 04:59:59.326225 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:59:59.326225 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:59:59.326225 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:59:59.326225 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:59:59.326225 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:59:59.326225 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:59:59.326225 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:59:59.326225 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jun 18 04:59:59.326225 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:59:59.326225 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:59:59.326225 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:59:59.326225 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:59:59.327819 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:59:59.327819 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:59:59.327819 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:59:59.327819 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:59:59.327819 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 18 04:59:59.327819 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 18 04:59:59.327819 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 18 04:59:59.327819 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 18 04:59:59.327819 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 18 04:59:59.327819 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 18 04:59:59.327819 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 18 04:59:59.327819 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 04:59:59.328592 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 526/1069] 10.222.0.23 () {56 vars in 1133 bytes} [Sat Jun 18 04:59:59 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:59.328592 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f30525-db97-4072-9bb9-f72878c2576c None None] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:59.328992 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 544/1070] 10.222.0.23 () {56 vars in 1096 bytes} [Sat Jun 18 04:59:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:59:59.329066 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f30525-db97-4072-9bb9-f72878c2576c None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:59.329066 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f30525-db97-4072-9bb9-f72878c2576c None None] PATH_INFO: `/` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:59.329066 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 527/1071] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 04:59:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 04:59:59.334013 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a196ae33-c352-40ba-b0aa-ea741d4dc59f None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:59.334405 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a196ae33-c352-40ba-b0aa-ea741d4dc59f None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:59.334838 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f30525-db97-4072-9bb9-f72878c2576c None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:59.334952 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a196ae33-c352-40ba-b0aa-ea741d4dc59f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:59.345881 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:59.351092 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-a196ae33-c352-40ba-b0aa-ea741d4dc59f None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 04:59:59.352476 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-a196ae33-c352-40ba-b0aa-ea741d4dc59f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 04:59:59.421297 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 545/1072] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 04:59:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 18 04:59:59.422243 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a196ae33-c352-40ba-b0aa-ea741d4dc59f None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:59.444906 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f30525-db97-4072-9bb9-f72878c2576c None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:59.478178 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea96639d-349d-47af-a81a-31bf8189340e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:59.478954 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea96639d-349d-47af-a81a-31bf8189340e None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:59.479230 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea96639d-349d-47af-a81a-31bf8189340e None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:59.479514 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea96639d-349d-47af-a81a-31bf8189340e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:59.490056 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea96639d-349d-47af-a81a-31bf8189340e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:59.492089 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea96639d-349d-47af-a81a-31bf8189340e None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:59.509849 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application [None req-ea96639d-349d-47af-a81a-31bf8189340e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 04:59:59.509849 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:59.509849 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:59:59.509849 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:59:59.509849 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:59:59.509849 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:59:59.509849 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:59:59.509849 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:59:59.509849 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:59:59.509849 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:59:59.509849 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:59:59.509849 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:59:59.509849 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 18 04:59:59.509849 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 18 04:59:59.509849 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 18 04:59:59.510761 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 18 04:59:59.510761 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:59:59.510761 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:59:59.510761 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:59:59.510761 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:59:59.510761 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:59:59.510761 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:59:59.510761 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:59:59.510761 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application with Lock( Jun 18 04:59:59.510761 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:59:59.510761 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:59:59.510761 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:59:59.510761 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:59:59.510761 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:59:59.510761 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:59:59.510761 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:59:59.510761 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:59:59.511660 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 18 04:59:59.511660 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 18 04:59:59.511660 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 04:59:59.511660 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 04:59:59.511660 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 528/1073] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Jun 18 04:59:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 18 04:59:59.512191 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea96639d-349d-47af-a81a-31bf8189340e None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:59.527405 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a196ae33-c352-40ba-b0aa-ea741d4dc59f None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:59.534659 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:59.561699 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38e6347-7880-4951-9ada-7b2d221cb9f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:59.562294 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38e6347-7880-4951-9ada-7b2d221cb9f2 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:59.562578 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38e6347-7880-4951-9ada-7b2d221cb9f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:59.562853 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38e6347-7880-4951-9ada-7b2d221cb9f2 None admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:59.563606 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38e6347-7880-4951-9ada-7b2d221cb9f2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:59.563920 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38e6347-7880-4951-9ada-7b2d221cb9f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:59.565288 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38e6347-7880-4951-9ada-7b2d221cb9f2 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:59.571077 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:59.573635 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 546/1074] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Jun 18 04:59:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:59.574412 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38e6347-7880-4951-9ada-7b2d221cb9f2 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:59.614751 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea96639d-349d-47af-a81a-31bf8189340e None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:59.649357 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e985ed-29de-4e5f-9af2-83de4bc1e3cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:59.650023 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e985ed-29de-4e5f-9af2-83de4bc1e3cb None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:59.650313 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e985ed-29de-4e5f-9af2-83de4bc1e3cb None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:59.650590 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e985ed-29de-4e5f-9af2-83de4bc1e3cb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:59.673474 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38e6347-7880-4951-9ada-7b2d221cb9f2 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:59.707516 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5589b9d7-bf8d-4c75-a6a8-137184bad2ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:59.708173 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5589b9d7-bf8d-4c75-a6a8-137184bad2ea None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:59.709034 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5589b9d7-bf8d-4c75-a6a8-137184bad2ea None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:59.709315 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5589b9d7-bf8d-4c75-a6a8-137184bad2ea None admin] PATH_INFO: `/v3/domains` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:59.710077 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5589b9d7-bf8d-4c75-a6a8-137184bad2ea None admin] RBAC: Adding query filter params (name=Default) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 04:59:59.710390 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5589b9d7-bf8d-4c75-a6a8-137184bad2ea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:59.711943 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5589b9d7-bf8d-4c75-a6a8-137184bad2ea None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:59.718714 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e985ed-29de-4e5f-9af2-83de4bc1e3cb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:59.720179 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e985ed-29de-4e5f-9af2-83de4bc1e3cb None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:59.724017 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 547/1075] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Jun 18 04:59:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:59.724839 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5589b9d7-bf8d-4c75-a6a8-137184bad2ea None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:59.754788 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:59.780901 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:59.824696 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 529/1076] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Jun 18 04:59:59 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 18 04:59:59.825470 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e985ed-29de-4e5f-9af2-83de4bc1e3cb None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:59.833500 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5589b9d7-bf8d-4c75-a6a8-137184bad2ea None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:59.868977 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40065fbb-9aa5-4dfe-b4a4-722e2d3720c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:59.869865 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40065fbb-9aa5-4dfe-b4a4-722e2d3720c8 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:59.870144 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40065fbb-9aa5-4dfe-b4a4-722e2d3720c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:59.870418 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40065fbb-9aa5-4dfe-b4a4-722e2d3720c8 None admin] PATH_INFO: `/v3/users/25348480076647289315bab6a97269a0` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:59.871607 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40065fbb-9aa5-4dfe-b4a4-722e2d3720c8 None admin] RBAC: Authorizing `identity:get_user(user_id=25348480076647289315bab6a97269a0)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:59.873038 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40065fbb-9aa5-4dfe-b4a4-722e2d3720c8 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:59.879758 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 548/1077] 10.222.0.23 () {56 vars in 1177 bytes} [Sat Jun 18 04:59:59 2022] GET /identity/v3/users/25348480076647289315bab6a97269a0 => generated 285 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 04:59:59.879758 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40065fbb-9aa5-4dfe-b4a4-722e2d3720c8 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 04:59:59.931546 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e985ed-29de-4e5f-9af2-83de4bc1e3cb None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:59.965025 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7004a5f-3856-4a4f-9f4f-d23a4a2747a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 04:59:59.965836 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7004a5f-3856-4a4f-9f4f-d23a4a2747a2 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 04:59:59.966121 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7004a5f-3856-4a4f-9f4f-d23a4a2747a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 04:59:59.966392 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7004a5f-3856-4a4f-9f4f-d23a4a2747a2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 04:59:59.968176 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 04:59:59.972926 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7004a5f-3856-4a4f-9f4f-d23a4a2747a2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 04:59:59.973794 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40065fbb-9aa5-4dfe-b4a4-722e2d3720c8 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 04:59:59.974934 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7004a5f-3856-4a4f-9f4f-d23a4a2747a2 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 04:59:59.988396 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application [None req-e7004a5f-3856-4a4f-9f4f-d23a4a2747a2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 18 04:59:59.988396 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 04:59:59.988396 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 04:59:59.988396 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 04:59:59.988396 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 04:59:59.988396 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 04:59:59.988396 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 04:59:59.988396 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 04:59:59.988396 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 04:59:59.988396 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 04:59:59.988396 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 04:59:59.988396 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 04:59:59.988396 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 18 04:59:59.988396 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 18 04:59:59.988396 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 18 04:59:59.989510 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 18 04:59:59.989510 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 04:59:59.989510 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 04:59:59.989510 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 04:59:59.989510 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 04:59:59.989510 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 04:59:59.989510 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 04:59:59.989510 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 04:59:59.989510 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application with Lock( Jun 18 04:59:59.989510 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 04:59:59.989510 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._enter() Jun 18 04:59:59.989510 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 04:59:59.989510 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 04:59:59.989510 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 04:59:59.989510 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.creator() Jun 18 04:59:59.989510 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 04:59:59.989510 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application created_value = creator( Jun 18 04:59:59.990534 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 18 04:59:59.990534 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 18 04:59:59.990534 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 18 04:59:59.990534 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 18 04:59:59.990534 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 18 04:59:59.990534 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 18 04:59:59.990534 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 18 04:59:59.990534 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 04:59:59.990534 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 530/1078] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Jun 18 04:59:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 04:59:59.991013 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7004a5f-3856-4a4f-9f4f-d23a4a2747a2 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:00:00.013310 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9c9cc5-ac71-4b8e-871e-24ab89294d13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:00:00.014141 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9c9cc5-ac71-4b8e-871e-24ab89294d13 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:00:00.014425 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9c9cc5-ac71-4b8e-871e-24ab89294d13 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:00:00.014706 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9c9cc5-ac71-4b8e-871e-24ab89294d13 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:00:00.015793 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:00:00.025594 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9c9cc5-ac71-4b8e-871e-24ab89294d13 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:00:00.027053 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9c9cc5-ac71-4b8e-871e-24ab89294d13 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:00:00.045139 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-9c9c9cc5-ac71-4b8e-871e-24ab89294d13 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 05:00:00.045139 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 05:00:00.045139 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 05:00:00.045139 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 05:00:00.045139 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 05:00:00.045139 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 05:00:00.045139 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 05:00:00.045139 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 05:00:00.045139 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 05:00:00.045139 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 05:00:00.045139 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 05:00:00.045139 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 05:00:00.045139 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 18 05:00:00.045139 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 18 05:00:00.045139 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 18 05:00:00.046104 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 18 05:00:00.046104 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 05:00:00.046104 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 05:00:00.046104 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 05:00:00.046104 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 05:00:00.046104 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 05:00:00.046104 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 05:00:00.046104 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 05:00:00.046104 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jun 18 05:00:00.046104 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 05:00:00.046104 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jun 18 05:00:00.046104 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 05:00:00.046104 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 05:00:00.046104 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 05:00:00.046104 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jun 18 05:00:00.046104 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 05:00:00.046104 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jun 18 05:00:00.047079 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 18 05:00:00.047079 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 18 05:00:00.047079 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 05:00:00.047079 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 05:00:00.047079 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 549/1079] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Jun 18 04:59:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 05:00:00.047377 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9c9cc5-ac71-4b8e-871e-24ab89294d13 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:00:00.098602 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7004a5f-3856-4a4f-9f4f-d23a4a2747a2 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:00:00.133129 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e75427-6104-49da-8e4f-9dd131939c3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:00:00.134004 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e75427-6104-49da-8e4f-9dd131939c3f None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:00:00.134352 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e75427-6104-49da-8e4f-9dd131939c3f None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:00:00.134629 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e75427-6104-49da-8e4f-9dd131939c3f None admin] PATH_INFO: `/v3/projects/8c469baa2dc5484abd09e54922200ffd` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:00:00.135814 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e75427-6104-49da-8e4f-9dd131939c3f None admin] RBAC: Authorizing `identity:get_project(project_id=8c469baa2dc5484abd09e54922200ffd)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:00:00.137241 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e75427-6104-49da-8e4f-9dd131939c3f None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:00:00.138972 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 531/1080] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Jun 18 04:59:59 2022] GET /identity/v3/projects/8c469baa2dc5484abd09e54922200ffd => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 05:00:00.139726 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e75427-6104-49da-8e4f-9dd131939c3f None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:00:00.148606 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9c9cc5-ac71-4b8e-871e-24ab89294d13 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:00:00.183401 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae8baea-adac-474f-a79f-ac2bef223483 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:00:00.184046 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae8baea-adac-474f-a79f-ac2bef223483 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:00:00.184328 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae8baea-adac-474f-a79f-ac2bef223483 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:00:00.184603 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae8baea-adac-474f-a79f-ac2bef223483 None admin] PATH_INFO: `/v3/projects` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:00:00.185411 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae8baea-adac-474f-a79f-ac2bef223483 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 05:00:00.185717 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae8baea-adac-474f-a79f-ac2bef223483 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:00:00.187152 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae8baea-adac-474f-a79f-ac2bef223483 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:00:00.198758 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 550/1081] 10.222.0.23 () {56 vars in 1148 bytes} [Sat Jun 18 05:00:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 05:00:00.199527 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae8baea-adac-474f-a79f-ac2bef223483 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:00:00.210741 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:00:00.234687 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:00:00.246756 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e75427-6104-49da-8e4f-9dd131939c3f None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:00:00.279615 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5739cb04-8b19-4ed7-861f-a026058c9078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:00:00.280271 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5739cb04-8b19-4ed7-861f-a026058c9078 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:00:00.280555 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5739cb04-8b19-4ed7-861f-a026058c9078 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:00:00.280832 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5739cb04-8b19-4ed7-861f-a026058c9078 None admin] PATH_INFO: `/v3/domains` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:00:00.281581 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5739cb04-8b19-4ed7-861f-a026058c9078 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 05:00:00.281895 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5739cb04-8b19-4ed7-861f-a026058c9078 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:00:00.283249 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5739cb04-8b19-4ed7-861f-a026058c9078 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:00:00.295167 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 532/1082] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Jun 18 05:00:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 05:00:00.295961 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5739cb04-8b19-4ed7-861f-a026058c9078 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:00:00.315770 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae8baea-adac-474f-a79f-ac2bef223483 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:00:00.359222 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6496e041-0f49-44d6-89fb-f3a0163c910d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:00:00.360036 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6496e041-0f49-44d6-89fb-f3a0163c910d None admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:00:00.360319 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6496e041-0f49-44d6-89fb-f3a0163c910d None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:00:00.360594 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6496e041-0f49-44d6-89fb-f3a0163c910d None admin] PATH_INFO: `/v3/projects/8c469baa2dc5484abd09e54922200ffd/users/25348480076647289315bab6a97269a0/roles/21c65f2c51bd4dd283d13507d2714467` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:00:00.362764 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6496e041-0f49-44d6-89fb-f3a0163c910d None admin] RBAC: Authorizing `identity:create_grant(project_id=8c469baa2dc5484abd09e54922200ffd, user_id=25348480076647289315bab6a97269a0, role_id=21c65f2c51bd4dd283d13507d2714467)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:00:00.364239 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6496e041-0f49-44d6-89fb-f3a0163c910d None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:00:00.415008 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5739cb04-8b19-4ed7-861f-a026058c9078 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:00:00.443279 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 551/1083] 10.222.0.23 () {58 vars in 1439 bytes} [Sat Jun 18 05:00:00 2022] PUT /identity/v3/projects/8c469baa2dc5484abd09e54922200ffd/users/25348480076647289315bab6a97269a0/roles/21c65f2c51bd4dd283d13507d2714467 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:00:00.455642 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d8c9df-ed27-4122-8e92-d6c11a19e909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:00:00.456585 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d8c9df-ed27-4122-8e92-d6c11a19e909 None admin] REQUEST_METHOD: `PUT` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:00:00.456916 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d8c9df-ed27-4122-8e92-d6c11a19e909 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:00:00.457193 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d8c9df-ed27-4122-8e92-d6c11a19e909 None admin] PATH_INFO: `/v3/projects/c95f8f3084c5472cb8364b7c57207394/users/15e7f45ee0744c1d9602187eac8e8b80/roles/2495da35026945aabd29678096996e70` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:00:00.460115 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d8c9df-ed27-4122-8e92-d6c11a19e909 None admin] RBAC: Authorizing `identity:create_grant(project_id=c95f8f3084c5472cb8364b7c57207394, user_id=15e7f45ee0744c1d9602187eac8e8b80, role_id=2495da35026945aabd29678096996e70)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:00:00.461701 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d8c9df-ed27-4122-8e92-d6c11a19e909 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:00:00.535122 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 533/1084] 10.222.0.23 () {58 vars in 1439 bytes} [Sat Jun 18 05:00:00 2022] PUT /identity/v3/projects/c95f8f3084c5472cb8364b7c57207394/users/15e7f45ee0744c1d9602187eac8e8b80/roles/2495da35026945aabd29678096996e70 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:00:02.514913 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49cf596-147d-4987-a75a-c166449b3218 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:00:02.515450 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49cf596-147d-4987-a75a-c166449b3218 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:00:02.515450 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49cf596-147d-4987-a75a-c166449b3218 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:00:02.516847 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 552/1085] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 05:00:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 05:00:02.522803 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a9fc03-6bac-4146-9084-1a460fefa772 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:00:02.523197 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a9fc03-6bac-4146-9084-1a460fefa772 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:00:02.523353 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a9fc03-6bac-4146-9084-1a460fefa772 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:00:02.540420 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-b2a9fc03-6bac-4146-9084-1a460fefa772 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:00:02.541613 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-b2a9fc03-6bac-4146-9084-1a460fefa772 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:00:02.543585 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:00:02.570321 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756ed30e-c6c2-45d1-b52f-a4da3ff302e2 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:00:02.570614 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756ed30e-c6c2-45d1-b52f-a4da3ff302e2 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:00:02.570914 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756ed30e-c6c2-45d1-b52f-a4da3ff302e2 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:00:02.572276 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 553/1086] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 05:00:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 18 05:00:02.577145 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8948f613-3981-42cd-9454-1893652eab2a None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:00:02.577463 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8948f613-3981-42cd-9454-1893652eab2a None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:00:02.577745 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8948f613-3981-42cd-9454-1893652eab2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:00:02.594282 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-8948f613-3981-42cd-9454-1893652eab2a None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:00:02.595480 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-8948f613-3981-42cd-9454-1893652eab2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:00:02.608088 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 534/1087] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 05:00:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 18 05:00:02.632792 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a9fc03-6bac-4146-9084-1a460fefa772 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:00:02.659270 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:00:02.664930 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 554/1088] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 05:00:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 18 05:00:02.688963 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8948f613-3981-42cd-9454-1893652eab2a None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:00:02.732551 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a9fc03-6bac-4146-9084-1a460fefa772 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:00:02.768721 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40ba24d-6021-4cad-8fb9-d13a300273a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:00:02.769535 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40ba24d-6021-4cad-8fb9-d13a300273a8 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:00:02.769821 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40ba24d-6021-4cad-8fb9-d13a300273a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:00:02.770093 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40ba24d-6021-4cad-8fb9-d13a300273a8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:00:02.777007 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:00:02.780813 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40ba24d-6021-4cad-8fb9-d13a300273a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 18 05:00:02.789164 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application [None req-f40ba24d-6021-4cad-8fb9-d13a300273a8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 18 05:00:02.789164 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 05:00:02.789164 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 05:00:02.789164 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 05:00:02.789164 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 05:00:02.789164 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 05:00:02.789164 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 05:00:02.789164 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 05:00:02.789164 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 05:00:02.789164 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 05:00:02.789164 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 05:00:02.789164 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 05:00:02.789164 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 18 05:00:02.789164 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 18 05:00:02.789164 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 18 05:00:02.789164 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 18 05:00:02.790260 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 18 05:00:02.790260 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 18 05:00:02.790260 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 18 05:00:02.790260 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 18 05:00:02.790260 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 05:00:02.790260 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 05:00:02.790260 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 05:00:02.790260 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 05:00:02.790260 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 05:00:02.790260 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 05:00:02.790260 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 05:00:02.790260 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application with Lock( Jun 18 05:00:02.790260 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 05:00:02.790260 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._enter() Jun 18 05:00:02.790260 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 05:00:02.790260 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 05:00:02.791141 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 05:00:02.791141 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.creator() Jun 18 05:00:02.791141 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 05:00:02.791141 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application created_value = creator( Jun 18 05:00:02.791141 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 18 05:00:02.791141 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 18 05:00:02.791141 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 18 05:00:02.791141 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 18 05:00:02.791141 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 18 05:00:02.791141 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 18 05:00:02.791141 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 18 05:00:02.791141 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 05:00:02.791141 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 535/1089] 10.222.0.23 () {56 vars in 1096 bytes} [Sat Jun 18 05:00:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 18 05:00:02.791824 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8948f613-3981-42cd-9454-1893652eab2a None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:00:02.796425 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40ba24d-6021-4cad-8fb9-d13a300273a8 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:00:02.825507 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44737084-d1ce-4dbf-8df8-eb7ffd932136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:00:02.826313 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44737084-d1ce-4dbf-8df8-eb7ffd932136 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:00:02.826597 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44737084-d1ce-4dbf-8df8-eb7ffd932136 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:00:02.826879 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44737084-d1ce-4dbf-8df8-eb7ffd932136 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:00:02.836918 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44737084-d1ce-4dbf-8df8-eb7ffd932136 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 18 05:00:02.845056 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-44737084-d1ce-4dbf-8df8-eb7ffd932136 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 18 05:00:02.845056 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 05:00:02.845056 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 05:00:02.845056 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 05:00:02.845056 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 05:00:02.845056 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 05:00:02.845056 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 05:00:02.845056 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 05:00:02.845056 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 05:00:02.845056 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 05:00:02.845056 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 05:00:02.845056 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 05:00:02.845056 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 18 05:00:02.845056 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 18 05:00:02.845056 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 18 05:00:02.845056 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 18 05:00:02.846152 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 18 05:00:02.846152 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 18 05:00:02.846152 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 18 05:00:02.846152 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 18 05:00:02.846152 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 05:00:02.846152 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 05:00:02.846152 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 05:00:02.846152 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 05:00:02.846152 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 05:00:02.846152 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 05:00:02.846152 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 05:00:02.846152 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jun 18 05:00:02.846152 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 05:00:02.846152 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jun 18 05:00:02.846152 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 05:00:02.846152 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 05:00:02.847003 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 05:00:02.847003 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jun 18 05:00:02.847003 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 05:00:02.847003 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jun 18 05:00:02.847003 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 18 05:00:02.847003 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 18 05:00:02.847003 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 18 05:00:02.847003 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 18 05:00:02.847003 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 18 05:00:02.847003 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 18 05:00:02.847003 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 18 05:00:02.847003 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 05:00:02.847003 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 555/1090] 10.222.0.23 () {56 vars in 1120 bytes} [Sat Jun 18 05:00:02 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 18 05:00:02.852951 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44737084-d1ce-4dbf-8df8-eb7ffd932136 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:00:02.898780 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40ba24d-6021-4cad-8fb9-d13a300273a8 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:00:02.912200 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:00:02.933939 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1192208e-96b0-401d-a564-d39a1e7b87c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:00:02.934602 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1192208e-96b0-401d-a564-d39a1e7b87c6 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:00:02.934890 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1192208e-96b0-401d-a564-d39a1e7b87c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:00:02.935168 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1192208e-96b0-401d-a564-d39a1e7b87c6 None admin] PATH_INFO: `/v3/roles` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:00:02.935907 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1192208e-96b0-401d-a564-d39a1e7b87c6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 05:00:02.936221 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1192208e-96b0-401d-a564-d39a1e7b87c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:00:02.937725 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1192208e-96b0-401d-a564-d39a1e7b87c6 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:00:02.946097 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 536/1091] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Jun 18 05:00:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 18 05:00:02.951767 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1192208e-96b0-401d-a564-d39a1e7b87c6 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:00:02.954568 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44737084-d1ce-4dbf-8df8-eb7ffd932136 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:00:02.988377 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e676262d-978b-4fc4-9968-d0042adbbdd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:00:02.989031 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e676262d-978b-4fc4-9968-d0042adbbdd9 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:00:02.989313 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e676262d-978b-4fc4-9968-d0042adbbdd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:00:02.989642 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e676262d-978b-4fc4-9968-d0042adbbdd9 None admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:00:02.990378 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e676262d-978b-4fc4-9968-d0042adbbdd9 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 05:00:02.990685 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e676262d-978b-4fc4-9968-d0042adbbdd9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:00:02.992295 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e676262d-978b-4fc4-9968-d0042adbbdd9 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:00:03.000115 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 556/1092] 10.222.0.23 () {56 vars in 1115 bytes} [Sat Jun 18 05:00:02 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 18 05:00:03.005581 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e676262d-978b-4fc4-9968-d0042adbbdd9 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:00:03.011090 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:00:03.051914 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1192208e-96b0-401d-a564-d39a1e7b87c6 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:00:03.084770 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a84106-e024-4729-8307-08d918e1670b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:00:03.085643 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a84106-e024-4729-8307-08d918e1670b None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:00:03.085920 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a84106-e024-4729-8307-08d918e1670b None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:00:03.086200 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a84106-e024-4729-8307-08d918e1670b None admin] PATH_INFO: `/v3/users/25348480076647289315bab6a97269a0` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:00:03.087381 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a84106-e024-4729-8307-08d918e1670b None admin] RBAC: Authorizing `identity:get_user(user_id=25348480076647289315bab6a97269a0)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:00:03.088791 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a84106-e024-4729-8307-08d918e1670b None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:00:03.090428 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 537/1093] 10.222.0.23 () {56 vars in 1177 bytes} [Sat Jun 18 05:00:02 2022] GET /identity/v3/users/25348480076647289315bab6a97269a0 => generated 285 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 18 05:00:03.096008 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a84106-e024-4729-8307-08d918e1670b None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:00:03.104184 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e676262d-978b-4fc4-9968-d0042adbbdd9 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:00:03.125737 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:00:03.139802 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f814e4ad-252b-4e44-8ea1-cad647718b12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:00:03.140583 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f814e4ad-252b-4e44-8ea1-cad647718b12 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:00:03.140863 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f814e4ad-252b-4e44-8ea1-cad647718b12 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:00:03.141135 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f814e4ad-252b-4e44-8ea1-cad647718b12 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:00:03.151710 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f814e4ad-252b-4e44-8ea1-cad647718b12 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:00:03.153149 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f814e4ad-252b-4e44-8ea1-cad647718b12 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:00:03.170385 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-f814e4ad-252b-4e44-8ea1-cad647718b12 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 05:00:03.170385 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 05:00:03.170385 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 05:00:03.170385 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 05:00:03.170385 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 05:00:03.170385 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 05:00:03.170385 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 05:00:03.170385 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 05:00:03.170385 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 05:00:03.170385 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 05:00:03.170385 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 05:00:03.170385 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 05:00:03.170385 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 18 05:00:03.170385 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 18 05:00:03.170385 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 18 05:00:03.171411 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 18 05:00:03.171411 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 05:00:03.171411 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 05:00:03.171411 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 05:00:03.171411 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 05:00:03.171411 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 05:00:03.171411 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 05:00:03.171411 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 05:00:03.171411 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jun 18 05:00:03.171411 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 05:00:03.171411 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jun 18 05:00:03.171411 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 05:00:03.171411 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 05:00:03.171411 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 05:00:03.171411 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jun 18 05:00:03.171411 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 05:00:03.171411 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jun 18 05:00:03.172656 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 18 05:00:03.172656 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 18 05:00:03.172656 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 05:00:03.172656 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 05:00:03.172656 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 557/1094] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Jun 18 05:00:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 18 05:00:03.177631 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f814e4ad-252b-4e44-8ea1-cad647718b12 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:00:03.198344 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a84106-e024-4729-8307-08d918e1670b None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:00:03.225706 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:00:03.233255 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3200a0-22eb-46f1-9d70-9246bf1bcb74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:00:03.233923 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3200a0-22eb-46f1-9d70-9246bf1bcb74 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:00:03.234202 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3200a0-22eb-46f1-9d70-9246bf1bcb74 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:00:03.234481 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3200a0-22eb-46f1-9d70-9246bf1bcb74 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:00:03.277856 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f814e4ad-252b-4e44-8ea1-cad647718b12 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:00:03.301578 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3200a0-22eb-46f1-9d70-9246bf1bcb74 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:00:03.303723 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3200a0-22eb-46f1-9d70-9246bf1bcb74 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:00:03.312023 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48556b77-6a73-4063-ae7b-24b13f3b5393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:00:03.312678 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48556b77-6a73-4063-ae7b-24b13f3b5393 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:00:03.312962 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48556b77-6a73-4063-ae7b-24b13f3b5393 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:00:03.313667 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48556b77-6a73-4063-ae7b-24b13f3b5393 None admin] PATH_INFO: `/v3/domains` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:00:03.314415 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48556b77-6a73-4063-ae7b-24b13f3b5393 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 05:00:03.315032 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48556b77-6a73-4063-ae7b-24b13f3b5393 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:00:03.316391 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48556b77-6a73-4063-ae7b-24b13f3b5393 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:00:03.328145 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 558/1095] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Jun 18 05:00:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 18 05:00:03.334154 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48556b77-6a73-4063-ae7b-24b13f3b5393 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:00:03.363908 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:00:03.408598 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 538/1096] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Jun 18 05:00:03 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 18 05:00:03.409863 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3200a0-22eb-46f1-9d70-9246bf1bcb74 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:00:03.432433 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48556b77-6a73-4063-ae7b-24b13f3b5393 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:00:03.436587 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:00:03.467006 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5257559a-f3f7-4969-ae26-4d6e4d92b62a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:00:03.467794 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5257559a-f3f7-4969-ae26-4d6e4d92b62a None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:00:03.468082 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5257559a-f3f7-4969-ae26-4d6e4d92b62a None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:00:03.468358 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5257559a-f3f7-4969-ae26-4d6e4d92b62a None admin] PATH_INFO: `/v3/users/nova` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:00:03.476042 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5257559a-f3f7-4969-ae26-4d6e4d92b62a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:00:03.477518 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5257559a-f3f7-4969-ae26-4d6e4d92b62a None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:00:03.492041 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-5257559a-f3f7-4969-ae26-4d6e4d92b62a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 18 05:00:03.492041 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 05:00:03.492041 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 05:00:03.492041 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 05:00:03.492041 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 05:00:03.492041 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 05:00:03.492041 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 05:00:03.492041 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 05:00:03.492041 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 05:00:03.492041 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 05:00:03.492041 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 05:00:03.492041 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 05:00:03.492041 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 18 05:00:03.492041 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 18 05:00:03.492041 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 18 05:00:03.492041 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 18 05:00:03.493055 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 05:00:03.493055 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 05:00:03.493055 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 18 05:00:03.493055 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 18 05:00:03.493055 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 18 05:00:03.493055 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 18 05:00:03.493055 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 05:00:03.493055 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 05:00:03.493055 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 05:00:03.493055 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 05:00:03.493055 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 05:00:03.493055 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jun 18 05:00:03.493055 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 05:00:03.493055 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jun 18 05:00:03.493055 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 05:00:03.493055 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 05:00:03.493055 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 05:00:03.493055 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jun 18 05:00:03.494028 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 05:00:03.494028 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jun 18 05:00:03.494028 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 18 05:00:03.494028 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 18 05:00:03.494028 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 18 05:00:03.494028 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 18 05:00:03.494028 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 18 05:00:03.494028 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 18 05:00:03.494028 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 18 05:00:03.494028 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 05:00:03.494028 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 559/1097] 10.222.0.23 () {56 vars in 1093 bytes} [Sat Jun 18 05:00:03 2022] GET /identity/v3/users/nova => generated 82 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 18 05:00:03.498824 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5257559a-f3f7-4969-ae26-4d6e4d92b62a None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:00:03.511607 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3200a0-22eb-46f1-9d70-9246bf1bcb74 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:00:03.546886 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4deb8a8d-098f-4559-ba4b-94281858abd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:00:03.547734 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deb8a8d-098f-4559-ba4b-94281858abd0 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:00:03.548017 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deb8a8d-098f-4559-ba4b-94281858abd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:00:03.548292 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deb8a8d-098f-4559-ba4b-94281858abd0 None admin] PATH_INFO: `/v3/projects/8c469baa2dc5484abd09e54922200ffd` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:00:03.549502 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4deb8a8d-098f-4559-ba4b-94281858abd0 None admin] RBAC: Authorizing `identity:get_project(project_id=8c469baa2dc5484abd09e54922200ffd)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:00:03.550923 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4deb8a8d-098f-4559-ba4b-94281858abd0 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:00:03.552660 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 539/1098] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Jun 18 05:00:03 2022] GET /identity/v3/projects/8c469baa2dc5484abd09e54922200ffd => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 18 05:00:03.558567 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4deb8a8d-098f-4559-ba4b-94281858abd0 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:00:03.590720 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:00:03.599629 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5257559a-f3f7-4969-ae26-4d6e4d92b62a None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:00:03.636062 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e057f11d-4f7b-482b-8b81-fd9f508f88c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:00:03.636716 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e057f11d-4f7b-482b-8b81-fd9f508f88c5 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:00:03.637003 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e057f11d-4f7b-482b-8b81-fd9f508f88c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:00:03.637274 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e057f11d-4f7b-482b-8b81-fd9f508f88c5 None admin] PATH_INFO: `/v3/users` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:00:03.638125 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e057f11d-4f7b-482b-8b81-fd9f508f88c5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 05:00:03.638433 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e057f11d-4f7b-482b-8b81-fd9f508f88c5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:00:03.639845 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e057f11d-4f7b-482b-8b81-fd9f508f88c5 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:00:03.654464 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:00:03.657798 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 560/1099] 10.222.0.23 () {56 vars in 1133 bytes} [Sat Jun 18 05:00:03 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 18 05:00:03.660284 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4deb8a8d-098f-4559-ba4b-94281858abd0 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:00:03.665038 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e057f11d-4f7b-482b-8b81-fd9f508f88c5 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:00:03.696464 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc86b081-60cf-46c9-81ca-48426869e5fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:00:03.697045 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc86b081-60cf-46c9-81ca-48426869e5fc None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:00:03.697326 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc86b081-60cf-46c9-81ca-48426869e5fc None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:00:03.697633 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc86b081-60cf-46c9-81ca-48426869e5fc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:00:03.698458 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc86b081-60cf-46c9-81ca-48426869e5fc None admin] RBAC: Adding query filter params (role.id=21c65f2c51bd4dd283d13507d2714467, scope.project.id=8c469baa2dc5484abd09e54922200ffd, user.id=25348480076647289315bab6a97269a0) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 05:00:03.698776 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc86b081-60cf-46c9-81ca-48426869e5fc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:00:03.700194 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc86b081-60cf-46c9-81ca-48426869e5fc None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:00:03.705933 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 540/1100] 10.222.0.23 () {56 vars in 1374 bytes} [Sat Jun 18 05:00:03 2022] GET /identity/v3/role_assignments?user.id=25348480076647289315bab6a97269a0&scope.project.id=8c469baa2dc5484abd09e54922200ffd&role.id=21c65f2c51bd4dd283d13507d2714467 => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 18 05:00:03.766362 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e057f11d-4f7b-482b-8b81-fd9f508f88c5 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:00:03.801869 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acca4857-4455-43bd-9df4-7dd7cce09485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:00:03.803321 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acca4857-4455-43bd-9df4-7dd7cce09485 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:00:03.803607 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acca4857-4455-43bd-9df4-7dd7cce09485 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:00:03.803883 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acca4857-4455-43bd-9df4-7dd7cce09485 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:00:03.825085 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:00:03.877719 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acca4857-4455-43bd-9df4-7dd7cce09485 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:00:03.879181 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acca4857-4455-43bd-9df4-7dd7cce09485 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:00:03.977108 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 561/1101] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Jun 18 05:00:03 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 18 05:00:03.982759 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc86b081-60cf-46c9-81ca-48426869e5fc None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:00:04.082495 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc86b081-60cf-46c9-81ca-48426869e5fc None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:00:04.116611 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6930ff68-e505-47bf-aaa3-55e01d41da8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:00:04.117676 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6930ff68-e505-47bf-aaa3-55e01d41da8f None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:00:04.117961 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6930ff68-e505-47bf-aaa3-55e01d41da8f None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:00:04.118235 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6930ff68-e505-47bf-aaa3-55e01d41da8f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:00:04.127954 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6930ff68-e505-47bf-aaa3-55e01d41da8f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:00:04.129376 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6930ff68-e505-47bf-aaa3-55e01d41da8f None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:00:04.147922 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application [None req-6930ff68-e505-47bf-aaa3-55e01d41da8f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 05:00:04.147922 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 05:00:04.147922 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 05:00:04.147922 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 05:00:04.147922 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 05:00:04.147922 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 05:00:04.147922 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 05:00:04.147922 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 05:00:04.147922 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 05:00:04.147922 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 05:00:04.147922 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 05:00:04.147922 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 05:00:04.147922 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 18 05:00:04.147922 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 18 05:00:04.147922 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 18 05:00:04.148838 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 18 05:00:04.148838 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 05:00:04.148838 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 05:00:04.148838 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 05:00:04.148838 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 05:00:04.148838 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 05:00:04.148838 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 05:00:04.148838 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 05:00:04.148838 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application with Lock( Jun 18 05:00:04.148838 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 05:00:04.148838 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._enter() Jun 18 05:00:04.148838 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 05:00:04.148838 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 05:00:04.148838 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 05:00:04.148838 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.creator() Jun 18 05:00:04.148838 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 05:00:04.148838 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application created_value = creator( Jun 18 05:00:04.149791 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 18 05:00:04.149791 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 18 05:00:04.149791 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 05:00:04.149791 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 05:00:04.149791 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 541/1102] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Jun 18 05:00:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 18 05:00:04.154762 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acca4857-4455-43bd-9df4-7dd7cce09485 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:00:04.204790 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:00:04.254672 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acca4857-4455-43bd-9df4-7dd7cce09485 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:00:04.288878 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85893cda-4940-4552-a920-418aa53c63d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:00:04.289573 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85893cda-4940-4552-a920-418aa53c63d2 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:00:04.289860 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85893cda-4940-4552-a920-418aa53c63d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:00:04.290138 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85893cda-4940-4552-a920-418aa53c63d2 None admin] PATH_INFO: `/v3/domains` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:00:04.290857 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85893cda-4940-4552-a920-418aa53c63d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 05:00:04.291167 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85893cda-4940-4552-a920-418aa53c63d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:00:04.292539 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85893cda-4940-4552-a920-418aa53c63d2 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:00:04.304169 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 562/1103] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Jun 18 05:00:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 05:00:04.309757 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6930ff68-e505-47bf-aaa3-55e01d41da8f None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:00:04.324298 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:00:04.409963 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6930ff68-e505-47bf-aaa3-55e01d41da8f None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:00:04.444867 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c7b408-67a0-4543-914d-5898800c37a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:00:04.445682 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c7b408-67a0-4543-914d-5898800c37a4 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:00:04.445966 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c7b408-67a0-4543-914d-5898800c37a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:00:04.446242 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c7b408-67a0-4543-914d-5898800c37a4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:00:04.452743 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c7b408-67a0-4543-914d-5898800c37a4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:00:04.454203 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c7b408-67a0-4543-914d-5898800c37a4 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:00:04.468123 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application [None req-88c7b408-67a0-4543-914d-5898800c37a4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 18 05:00:04.468123 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 05:00:04.468123 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 05:00:04.468123 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 05:00:04.468123 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 05:00:04.468123 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 05:00:04.468123 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 05:00:04.468123 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 05:00:04.468123 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 05:00:04.468123 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 05:00:04.468123 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 05:00:04.468123 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 05:00:04.468123 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 18 05:00:04.468123 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 18 05:00:04.468123 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 18 05:00:04.469004 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 18 05:00:04.469004 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 05:00:04.469004 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 05:00:04.469004 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 05:00:04.469004 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 05:00:04.469004 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 05:00:04.469004 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 05:00:04.469004 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 05:00:04.469004 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application with Lock( Jun 18 05:00:04.469004 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 05:00:04.469004 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._enter() Jun 18 05:00:04.469004 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 05:00:04.469004 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 05:00:04.469004 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 05:00:04.469004 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.creator() Jun 18 05:00:04.469004 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 05:00:04.469004 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application created_value = creator( Jun 18 05:00:04.470127 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 18 05:00:04.470127 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 18 05:00:04.470127 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 18 05:00:04.470127 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 18 05:00:04.470127 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 18 05:00:04.470127 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 18 05:00:04.470127 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 18 05:00:04.470127 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 05:00:04.470127 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 542/1104] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Jun 18 05:00:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 05:00:04.475620 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85893cda-4940-4552-a920-418aa53c63d2 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:00:04.574278 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85893cda-4940-4552-a920-418aa53c63d2 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:00:04.584259 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:00:04.606581 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46f39c0-9320-4c3c-ae54-543ccd8e46f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:00:04.607226 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46f39c0-9320-4c3c-ae54-543ccd8e46f5 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:00:04.607507 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46f39c0-9320-4c3c-ae54-543ccd8e46f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:00:04.607788 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46f39c0-9320-4c3c-ae54-543ccd8e46f5 None admin] PATH_INFO: `/v3/projects` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:00:04.608563 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46f39c0-9320-4c3c-ae54-543ccd8e46f5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 05:00:04.608886 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46f39c0-9320-4c3c-ae54-543ccd8e46f5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:00:04.610825 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46f39c0-9320-4c3c-ae54-543ccd8e46f5 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:00:04.624161 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 563/1105] 10.222.0.23 () {56 vars in 1148 bytes} [Sat Jun 18 05:00:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 05:00:04.630821 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c7b408-67a0-4543-914d-5898800c37a4 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:00:04.698908 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:00:04.728081 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c7b408-67a0-4543-914d-5898800c37a4 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:00:04.762651 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f469c9b1-56ee-4fb0-afe8-0a4bd483d704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:00:04.763302 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f469c9b1-56ee-4fb0-afe8-0a4bd483d704 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:00:04.763586 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f469c9b1-56ee-4fb0-afe8-0a4bd483d704 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:00:04.763864 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f469c9b1-56ee-4fb0-afe8-0a4bd483d704 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:00:04.764690 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f469c9b1-56ee-4fb0-afe8-0a4bd483d704 None admin] RBAC: Adding query filter params (role.id=2495da35026945aabd29678096996e70, scope.project.id=c95f8f3084c5472cb8364b7c57207394, user.id=15e7f45ee0744c1d9602187eac8e8b80) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 05:00:04.765009 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f469c9b1-56ee-4fb0-afe8-0a4bd483d704 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:00:04.766474 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f469c9b1-56ee-4fb0-afe8-0a4bd483d704 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:00:04.772386 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 543/1106] 10.222.0.23 () {56 vars in 1374 bytes} [Sat Jun 18 05:00:04 2022] GET /identity/v3/role_assignments?user.id=15e7f45ee0744c1d9602187eac8e8b80&scope.project.id=c95f8f3084c5472cb8364b7c57207394&role.id=2495da35026945aabd29678096996e70 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 05:00:05.777241 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fea57e-f280-47a1-ba15-bfdd30a8405a None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:00:05.777809 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fea57e-f280-47a1-ba15-bfdd30a8405a None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:00:05.777903 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fea57e-f280-47a1-ba15-bfdd30a8405a None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:00:05.779214 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 564/1107] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 05:00:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 05:00:05.784355 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0706a88b-7e1b-4c13-ad84-50c22da38b2e None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:00:05.784780 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0706a88b-7e1b-4c13-ad84-50c22da38b2e None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:00:05.784905 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0706a88b-7e1b-4c13-ad84-50c22da38b2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:00:05.801642 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-0706a88b-7e1b-4c13-ad84-50c22da38b2e None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:00:05.802849 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-0706a88b-7e1b-4c13-ad84-50c22da38b2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:00:05.867569 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 544/1108] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 05:00:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 18 05:00:05.892606 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fea57e-f280-47a1-ba15-bfdd30a8405a None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:00:06.000157 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fea57e-f280-47a1-ba15-bfdd30a8405a None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:00:06.031559 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c28d1f-67f9-4000-bb94-7677b9433f53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:00:06.032281 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c28d1f-67f9-4000-bb94-7677b9433f53 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:00:06.032564 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c28d1f-67f9-4000-bb94-7677b9433f53 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:00:06.032844 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c28d1f-67f9-4000-bb94-7677b9433f53 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:00:06.034035 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c28d1f-67f9-4000-bb94-7677b9433f53 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:00:06.035468 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c28d1f-67f9-4000-bb94-7677b9433f53 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:00:06.037167 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 565/1109] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Jun 18 05:00:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 05:00:06.042899 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0706a88b-7e1b-4c13-ad84-50c22da38b2e None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:00:06.107260 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:00:06.141789 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0706a88b-7e1b-4c13-ad84-50c22da38b2e None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:00:06.176545 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f368ed6-29ab-40ba-a26f-11174059d38a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:00:06.178109 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f368ed6-29ab-40ba-a26f-11174059d38a None admin] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:00:06.178396 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f368ed6-29ab-40ba-a26f-11174059d38a None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:00:06.178667 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f368ed6-29ab-40ba-a26f-11174059d38a None admin] PATH_INFO: `/v3/users` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:00:06.179379 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f368ed6-29ab-40ba-a26f-11174059d38a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:00:06.180798 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f368ed6-29ab-40ba-a26f-11174059d38a None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:00:06.265216 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 545/1110] 10.222.0.23 () {60 vars in 1132 bytes} [Sat Jun 18 05:00:06 2022] POST /identity/v3/users => generated 286 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:00:08.371849 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49579e3f-1564-45d6-8a89-9d8603ebfbfa None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:00:08.372438 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49579e3f-1564-45d6-8a89-9d8603ebfbfa None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:00:08.372438 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49579e3f-1564-45d6-8a89-9d8603ebfbfa None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:00:08.373847 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 566/1111] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 05:00:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 05:00:08.379208 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4954e9a-c9e3-4ad5-b9ac-aeec24038861 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:00:08.379635 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4954e9a-c9e3-4ad5-b9ac-aeec24038861 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:00:08.379758 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4954e9a-c9e3-4ad5-b9ac-aeec24038861 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:00:08.397573 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-d4954e9a-c9e3-4ad5-b9ac-aeec24038861 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:00:08.398767 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-d4954e9a-c9e3-4ad5-b9ac-aeec24038861 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:00:08.463789 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 546/1112] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 05:00:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 18 05:00:08.487402 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49579e3f-1564-45d6-8a89-9d8603ebfbfa None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:00:08.528346 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:00:08.587065 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49579e3f-1564-45d6-8a89-9d8603ebfbfa None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:00:08.619713 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0deb0962-b724-4c54-83ac-4e7bbfa3482a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:00:08.620556 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deb0962-b724-4c54-83ac-4e7bbfa3482a None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:00:08.620839 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deb0962-b724-4c54-83ac-4e7bbfa3482a None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:00:08.621114 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deb0962-b724-4c54-83ac-4e7bbfa3482a None admin] PATH_INFO: `/v3/roles/0089a3aad83c4ae08d8e9d864957285b` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:00:08.622657 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0deb0962-b724-4c54-83ac-4e7bbfa3482a None admin] RBAC: Authorizing `identity:get_role(role_id=0089a3aad83c4ae08d8e9d864957285b)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:00:08.623975 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0deb0962-b724-4c54-83ac-4e7bbfa3482a None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:00:08.625327 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 567/1113] 10.222.0.23 () {56 vars in 1177 bytes} [Sat Jun 18 05:00:08 2022] GET /identity/v3/roles/0089a3aad83c4ae08d8e9d864957285b => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 05:00:08.631549 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4954e9a-c9e3-4ad5-b9ac-aeec24038861 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:00:08.682584 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:00:08.731464 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4954e9a-c9e3-4ad5-b9ac-aeec24038861 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:00:08.764935 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1a70cb-b772-49aa-a361-b6dde1647461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:00:08.765819 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1a70cb-b772-49aa-a361-b6dde1647461 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:00:08.766102 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1a70cb-b772-49aa-a361-b6dde1647461 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:00:08.766375 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1a70cb-b772-49aa-a361-b6dde1647461 None admin] PATH_INFO: `/v3/users/34e633174258451f82ae3642f6f76be2` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:00:08.778717 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1a70cb-b772-49aa-a361-b6dde1647461 None admin] RBAC: Authorizing `identity:get_user(user_id=34e633174258451f82ae3642f6f76be2)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:00:08.780142 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1a70cb-b772-49aa-a361-b6dde1647461 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:00:08.781821 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 547/1114] 10.222.0.23 () {56 vars in 1177 bytes} [Sat Jun 18 05:00:08 2022] GET /identity/v3/users/34e633174258451f82ae3642f6f76be2 => generated 286 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 05:00:08.788204 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0deb0962-b724-4c54-83ac-4e7bbfa3482a None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:00:08.885825 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0deb0962-b724-4c54-83ac-4e7bbfa3482a None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:00:08.892760 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:00:08.918489 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84e4881-9f7b-4243-8610-06fa9bbdfe1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:00:08.919114 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84e4881-9f7b-4243-8610-06fa9bbdfe1d None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:00:08.919392 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84e4881-9f7b-4243-8610-06fa9bbdfe1d None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:00:08.919663 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84e4881-9f7b-4243-8610-06fa9bbdfe1d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:00:08.983795 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84e4881-9f7b-4243-8610-06fa9bbdfe1d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:00:08.985225 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84e4881-9f7b-4243-8610-06fa9bbdfe1d None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:00:09.079700 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 568/1115] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Jun 18 05:00:08 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 18 05:00:09.086207 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1a70cb-b772-49aa-a361-b6dde1647461 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:00:09.182655 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1a70cb-b772-49aa-a361-b6dde1647461 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:00:09.192508 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:00:09.216164 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1093ab73-cead-4de8-94ab-05b881fc6596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:00:09.217004 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1093ab73-cead-4de8-94ab-05b881fc6596 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:00:09.217287 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1093ab73-cead-4de8-94ab-05b881fc6596 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:00:09.217598 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1093ab73-cead-4de8-94ab-05b881fc6596 None admin] PATH_INFO: `/v3/projects/8c469baa2dc5484abd09e54922200ffd` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:00:09.218770 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1093ab73-cead-4de8-94ab-05b881fc6596 None admin] RBAC: Authorizing `identity:get_project(project_id=8c469baa2dc5484abd09e54922200ffd)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:00:09.220192 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1093ab73-cead-4de8-94ab-05b881fc6596 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:00:09.221912 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 548/1116] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Jun 18 05:00:09 2022] GET /identity/v3/projects/8c469baa2dc5484abd09e54922200ffd => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 05:00:09.229522 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84e4881-9f7b-4243-8610-06fa9bbdfe1d None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:00:09.237311 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:00:09.324400 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84e4881-9f7b-4243-8610-06fa9bbdfe1d None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:00:09.357219 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d899df60-489e-4ed7-91fc-7f324fea6834 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:00:09.357881 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d899df60-489e-4ed7-91fc-7f324fea6834 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:00:09.358163 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d899df60-489e-4ed7-91fc-7f324fea6834 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:00:09.358438 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d899df60-489e-4ed7-91fc-7f324fea6834 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:00:09.359248 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d899df60-489e-4ed7-91fc-7f324fea6834 None admin] RBAC: Adding query filter params (role.id=0089a3aad83c4ae08d8e9d864957285b, scope.project.id=8c469baa2dc5484abd09e54922200ffd, user.id=34e633174258451f82ae3642f6f76be2) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 05:00:09.359561 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d899df60-489e-4ed7-91fc-7f324fea6834 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:00:09.360975 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d899df60-489e-4ed7-91fc-7f324fea6834 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:00:09.366357 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 569/1117] 10.222.0.23 () {56 vars in 1374 bytes} [Sat Jun 18 05:00:09 2022] GET /identity/v3/role_assignments?user.id=34e633174258451f82ae3642f6f76be2&scope.project.id=8c469baa2dc5484abd09e54922200ffd&role.id=0089a3aad83c4ae08d8e9d864957285b => generated 259 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 05:00:11.385569 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b955dd-4279-46db-8b9a-509733f0085a None None] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:00:11.386113 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b955dd-4279-46db-8b9a-509733f0085a None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:00:11.386113 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b955dd-4279-46db-8b9a-509733f0085a None None] PATH_INFO: `/` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:00:11.387495 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 549/1118] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 05:00:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 05:00:11.392798 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c387f853-6f75-475d-a705-bc830ac52847 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:00:11.393171 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c387f853-6f75-475d-a705-bc830ac52847 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:00:11.393352 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c387f853-6f75-475d-a705-bc830ac52847 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:00:11.409755 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-c387f853-6f75-475d-a705-bc830ac52847 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:00:11.410938 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-c387f853-6f75-475d-a705-bc830ac52847 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:00:11.472057 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:00:11.476822 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 570/1119] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 05:00:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 18 05:00:11.500422 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b955dd-4279-46db-8b9a-509733f0085a None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:00:11.600564 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b955dd-4279-46db-8b9a-509733f0085a None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:00:11.633479 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2949f501-c038-4d54-94f9-b6057e3defdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:00:11.634317 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2949f501-c038-4d54-94f9-b6057e3defdd None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:00:11.634602 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2949f501-c038-4d54-94f9-b6057e3defdd None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:00:11.634880 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2949f501-c038-4d54-94f9-b6057e3defdd None admin] PATH_INFO: `/v3/roles/0089a3aad83c4ae08d8e9d864957285b` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:00:11.636396 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2949f501-c038-4d54-94f9-b6057e3defdd None admin] RBAC: Authorizing `identity:get_role(role_id=0089a3aad83c4ae08d8e9d864957285b)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:00:11.637745 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2949f501-c038-4d54-94f9-b6057e3defdd None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:00:11.639085 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 550/1120] 10.222.0.23 () {56 vars in 1177 bytes} [Sat Jun 18 05:00:11 2022] GET /identity/v3/roles/0089a3aad83c4ae08d8e9d864957285b => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 05:00:11.644843 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c387f853-6f75-475d-a705-bc830ac52847 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:00:11.740712 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c387f853-6f75-475d-a705-bc830ac52847 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:00:11.774309 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8d0f6a-a74f-4847-a6a1-33cbe33be253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:00:11.775158 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8d0f6a-a74f-4847-a6a1-33cbe33be253 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:00:11.775440 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8d0f6a-a74f-4847-a6a1-33cbe33be253 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:00:11.775714 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8d0f6a-a74f-4847-a6a1-33cbe33be253 None admin] PATH_INFO: `/v3/users/34e633174258451f82ae3642f6f76be2` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:00:11.776887 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8d0f6a-a74f-4847-a6a1-33cbe33be253 None admin] RBAC: Authorizing `identity:get_user(user_id=34e633174258451f82ae3642f6f76be2)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:00:11.778346 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8d0f6a-a74f-4847-a6a1-33cbe33be253 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:00:11.779938 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 571/1121] 10.222.0.23 () {56 vars in 1177 bytes} [Sat Jun 18 05:00:11 2022] GET /identity/v3/users/34e633174258451f82ae3642f6f76be2 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 05:00:11.785618 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2949f501-c038-4d54-94f9-b6057e3defdd None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:00:11.830911 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:00:11.883135 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2949f501-c038-4d54-94f9-b6057e3defdd None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:00:11.916021 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fbdaad-1db1-47c4-8a67-539175a2337b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:00:11.916798 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fbdaad-1db1-47c4-8a67-539175a2337b None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:00:11.917079 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fbdaad-1db1-47c4-8a67-539175a2337b None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:00:11.917350 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fbdaad-1db1-47c4-8a67-539175a2337b None admin] PATH_INFO: `/v3/projects/8c469baa2dc5484abd09e54922200ffd` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:00:11.918551 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fbdaad-1db1-47c4-8a67-539175a2337b None admin] RBAC: Authorizing `identity:get_project(project_id=8c469baa2dc5484abd09e54922200ffd)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:00:11.919960 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fbdaad-1db1-47c4-8a67-539175a2337b None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:00:11.921660 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 551/1122] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Jun 18 05:00:11 2022] GET /identity/v3/projects/8c469baa2dc5484abd09e54922200ffd => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 05:00:11.927901 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8d0f6a-a74f-4847-a6a1-33cbe33be253 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:00:11.993402 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:00:12.025358 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8d0f6a-a74f-4847-a6a1-33cbe33be253 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:00:12.058920 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834ba0cd-95e4-4688-8d70-55bc256e4207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:00:12.059786 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834ba0cd-95e4-4688-8d70-55bc256e4207 None admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:00:12.060070 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834ba0cd-95e4-4688-8d70-55bc256e4207 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:00:12.060344 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834ba0cd-95e4-4688-8d70-55bc256e4207 None admin] PATH_INFO: `/v3/projects/8c469baa2dc5484abd09e54922200ffd/users/34e633174258451f82ae3642f6f76be2/roles/0089a3aad83c4ae08d8e9d864957285b` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:00:12.062543 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834ba0cd-95e4-4688-8d70-55bc256e4207 None admin] RBAC: Authorizing `identity:create_grant(project_id=8c469baa2dc5484abd09e54922200ffd, user_id=34e633174258451f82ae3642f6f76be2, role_id=0089a3aad83c4ae08d8e9d864957285b)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:00:12.063970 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834ba0cd-95e4-4688-8d70-55bc256e4207 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:00:12.162219 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 572/1123] 10.222.0.23 () {58 vars in 1439 bytes} [Sat Jun 18 05:00:11 2022] PUT /identity/v3/projects/8c469baa2dc5484abd09e54922200ffd/users/34e633174258451f82ae3642f6f76be2/roles/0089a3aad83c4ae08d8e9d864957285b => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 18 05:00:14.149772 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28211863-b8ce-4dd2-85f0-bdaf8a87ce35 None None] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:00:14.150326 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28211863-b8ce-4dd2-85f0-bdaf8a87ce35 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:00:14.150326 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28211863-b8ce-4dd2-85f0-bdaf8a87ce35 None None] PATH_INFO: `/` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:00:14.151654 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 552/1124] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 05:00:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 05:00:14.156786 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d72bd23-414e-4ffc-adf0-563a3015b5d5 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:00:14.157217 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d72bd23-414e-4ffc-adf0-563a3015b5d5 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:00:14.157318 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d72bd23-414e-4ffc-adf0-563a3015b5d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:00:14.174425 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-1d72bd23-414e-4ffc-adf0-563a3015b5d5 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:00:14.175588 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-1d72bd23-414e-4ffc-adf0-563a3015b5d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:00:14.241692 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 573/1125] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 05:00:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 18 05:00:14.264953 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28211863-b8ce-4dd2-85f0-bdaf8a87ce35 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:00:14.365210 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28211863-b8ce-4dd2-85f0-bdaf8a87ce35 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:00:14.378688 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:00:14.398863 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8815e7a0-9cb7-41e9-8f21-1fac55c3faec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:00:14.401519 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8815e7a0-9cb7-41e9-8f21-1fac55c3faec None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:00:14.401519 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8815e7a0-9cb7-41e9-8f21-1fac55c3faec None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:00:14.401519 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8815e7a0-9cb7-41e9-8f21-1fac55c3faec None admin] PATH_INFO: `/v3/roles/0089a3aad83c4ae08d8e9d864957285b` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:00:14.401806 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8815e7a0-9cb7-41e9-8f21-1fac55c3faec None admin] RBAC: Authorizing `identity:get_role(role_id=0089a3aad83c4ae08d8e9d864957285b)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:00:14.403242 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8815e7a0-9cb7-41e9-8f21-1fac55c3faec None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:00:14.404606 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 553/1126] 10.222.0.23 () {56 vars in 1177 bytes} [Sat Jun 18 05:00:14 2022] GET /identity/v3/roles/0089a3aad83c4ae08d8e9d864957285b => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 05:00:14.410449 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d72bd23-414e-4ffc-adf0-563a3015b5d5 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:00:14.466770 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:00:14.508042 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d72bd23-414e-4ffc-adf0-563a3015b5d5 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:00:14.541654 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083941b0-0601-4271-aec0-c0a6e3eed93e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:00:14.542516 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083941b0-0601-4271-aec0-c0a6e3eed93e None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:00:14.542799 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083941b0-0601-4271-aec0-c0a6e3eed93e None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:00:14.543076 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083941b0-0601-4271-aec0-c0a6e3eed93e None admin] PATH_INFO: `/v3/users/34e633174258451f82ae3642f6f76be2` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:00:14.544258 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083941b0-0601-4271-aec0-c0a6e3eed93e None admin] RBAC: Authorizing `identity:get_user(user_id=34e633174258451f82ae3642f6f76be2)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:00:14.545734 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083941b0-0601-4271-aec0-c0a6e3eed93e None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:00:14.547365 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 574/1127] 10.222.0.23 () {56 vars in 1177 bytes} [Sat Jun 18 05:00:14 2022] GET /identity/v3/users/34e633174258451f82ae3642f6f76be2 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 18 05:00:14.554019 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8815e7a0-9cb7-41e9-8f21-1fac55c3faec None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:00:14.652112 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8815e7a0-9cb7-41e9-8f21-1fac55c3faec None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:00:14.685670 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c12f3fc-00ce-4aae-a198-580864d42115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:00:14.686296 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c12f3fc-00ce-4aae-a198-580864d42115 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:00:14.686576 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c12f3fc-00ce-4aae-a198-580864d42115 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:00:14.686847 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c12f3fc-00ce-4aae-a198-580864d42115 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:00:14.737440 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:00:14.756508 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c12f3fc-00ce-4aae-a198-580864d42115 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:00:14.758010 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c12f3fc-00ce-4aae-a198-580864d42115 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:00:14.854917 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 554/1128] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Jun 18 05:00:14 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 18 05:00:14.860843 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083941b0-0601-4271-aec0-c0a6e3eed93e None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:00:14.959547 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083941b0-0601-4271-aec0-c0a6e3eed93e None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:00:14.984270 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:00:14.993473 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21747990-0a92-4fd5-b031-a844c1a8fd90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:00:14.996328 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21747990-0a92-4fd5-b031-a844c1a8fd90 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:00:14.996328 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21747990-0a92-4fd5-b031-a844c1a8fd90 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:00:14.996328 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21747990-0a92-4fd5-b031-a844c1a8fd90 None admin] PATH_INFO: `/v3/projects/8c469baa2dc5484abd09e54922200ffd` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:00:14.996328 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21747990-0a92-4fd5-b031-a844c1a8fd90 None admin] RBAC: Authorizing `identity:get_project(project_id=8c469baa2dc5484abd09e54922200ffd)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:00:14.997543 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21747990-0a92-4fd5-b031-a844c1a8fd90 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:00:14.999226 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 575/1129] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Jun 18 05:00:14 2022] GET /identity/v3/projects/8c469baa2dc5484abd09e54922200ffd => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 05:00:15.006185 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c12f3fc-00ce-4aae-a198-580864d42115 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:00:15.085898 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:00:15.103911 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c12f3fc-00ce-4aae-a198-580864d42115 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:00:15.137051 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e11978-f190-43d5-9d1b-4ede419641c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:00:15.137728 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e11978-f190-43d5-9d1b-4ede419641c6 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:00:15.138013 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e11978-f190-43d5-9d1b-4ede419641c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:00:15.138291 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e11978-f190-43d5-9d1b-4ede419641c6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:00:15.139113 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e11978-f190-43d5-9d1b-4ede419641c6 None admin] RBAC: Adding query filter params (role.id=0089a3aad83c4ae08d8e9d864957285b, scope.project.id=8c469baa2dc5484abd09e54922200ffd, user.id=34e633174258451f82ae3642f6f76be2) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 05:00:15.139426 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e11978-f190-43d5-9d1b-4ede419641c6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:00:15.140845 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e11978-f190-43d5-9d1b-4ede419641c6 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:00:15.147193 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 555/1130] 10.222.0.23 () {56 vars in 1374 bytes} [Sat Jun 18 05:00:15 2022] GET /identity/v3/role_assignments?user.id=34e633174258451f82ae3642f6f76be2&scope.project.id=8c469baa2dc5484abd09e54922200ffd&role.id=0089a3aad83c4ae08d8e9d864957285b => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 05:00:17.199894 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9856a4-28c8-422a-95b6-0bd875d6804e None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:00:17.200415 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9856a4-28c8-422a-95b6-0bd875d6804e None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:00:17.200526 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9856a4-28c8-422a-95b6-0bd875d6804e None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:00:17.201872 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 576/1131] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 05:00:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 05:00:17.207016 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa93eefc-5b14-4ee0-8507-8628370b9bd7 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:00:17.207413 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa93eefc-5b14-4ee0-8507-8628370b9bd7 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:00:17.207561 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa93eefc-5b14-4ee0-8507-8628370b9bd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:00:17.225719 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-aa93eefc-5b14-4ee0-8507-8628370b9bd7 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:00:17.226924 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-aa93eefc-5b14-4ee0-8507-8628370b9bd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:00:17.292563 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 556/1132] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 05:00:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 18 05:00:17.316166 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9856a4-28c8-422a-95b6-0bd875d6804e None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:00:17.415989 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9856a4-28c8-422a-95b6-0bd875d6804e None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:00:17.449402 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d00585a-0d2b-4547-9d00-8ba0b562bdf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:00:17.450208 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d00585a-0d2b-4547-9d00-8ba0b562bdf0 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:00:17.450442 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d00585a-0d2b-4547-9d00-8ba0b562bdf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:00:17.450719 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d00585a-0d2b-4547-9d00-8ba0b562bdf0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:00:17.451890 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d00585a-0d2b-4547-9d00-8ba0b562bdf0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:00:17.453320 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d00585a-0d2b-4547-9d00-8ba0b562bdf0 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:00:17.454952 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 577/1133] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Jun 18 05:00:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 05:00:17.461804 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa93eefc-5b14-4ee0-8507-8628370b9bd7 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:00:17.535902 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:00:17.558461 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa93eefc-5b14-4ee0-8507-8628370b9bd7 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:00:17.591474 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7244df9a-17d2-41be-9621-bab28543b175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:00:17.592115 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7244df9a-17d2-41be-9621-bab28543b175 None admin] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:00:17.592399 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7244df9a-17d2-41be-9621-bab28543b175 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:00:17.592674 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7244df9a-17d2-41be-9621-bab28543b175 None admin] PATH_INFO: `/v3/projects` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:00:17.593404 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7244df9a-17d2-41be-9621-bab28543b175 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:00:17.594828 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7244df9a-17d2-41be-9621-bab28543b175 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:00:17.679402 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 557/1134] 10.222.0.23 () {60 vars in 1141 bytes} [Sat Jun 18 05:00:17 2022] POST /identity/v3/projects => generated 311 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:00:19.729669 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0b5881-a18c-43c2-8a75-a126e40fe337 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:00:19.730207 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0b5881-a18c-43c2-8a75-a126e40fe337 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:00:19.730207 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0b5881-a18c-43c2-8a75-a126e40fe337 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:00:19.731584 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 578/1135] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 05:00:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 05:00:19.736779 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddcdf44-6aaf-4134-a727-7081bd2362b8 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:00:19.737195 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddcdf44-6aaf-4134-a727-7081bd2362b8 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:00:19.737329 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddcdf44-6aaf-4134-a727-7081bd2362b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:00:19.756613 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-cddcdf44-6aaf-4134-a727-7081bd2362b8 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:00:19.757825 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-cddcdf44-6aaf-4134-a727-7081bd2362b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:00:19.823940 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 558/1136] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 05:00:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 18 05:00:19.849253 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0b5881-a18c-43c2-8a75-a126e40fe337 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:00:19.910776 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:00:19.950294 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0b5881-a18c-43c2-8a75-a126e40fe337 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:00:19.983188 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104e8026-e718-4f2b-8eba-a917d9986186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:00:19.983907 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104e8026-e718-4f2b-8eba-a917d9986186 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:00:19.984188 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104e8026-e718-4f2b-8eba-a917d9986186 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:00:19.984458 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104e8026-e718-4f2b-8eba-a917d9986186 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:00:19.985654 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104e8026-e718-4f2b-8eba-a917d9986186 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:00:19.987082 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104e8026-e718-4f2b-8eba-a917d9986186 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:00:19.988684 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 579/1137] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Jun 18 05:00:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 05:00:19.995412 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddcdf44-6aaf-4134-a727-7081bd2362b8 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:00:20.059029 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:00:20.098443 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddcdf44-6aaf-4134-a727-7081bd2362b8 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:00:20.132846 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa048f67-3d83-47e6-a754-11e7a906299e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:00:20.133528 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa048f67-3d83-47e6-a754-11e7a906299e None admin] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:00:20.133812 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa048f67-3d83-47e6-a754-11e7a906299e None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:00:20.134087 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa048f67-3d83-47e6-a754-11e7a906299e None admin] PATH_INFO: `/v3/users` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:00:20.134785 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa048f67-3d83-47e6-a754-11e7a906299e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:00:20.136203 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa048f67-3d83-47e6-a754-11e7a906299e None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:00:20.231075 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 559/1138] 10.222.0.23 () {60 vars in 1132 bytes} [Sat Jun 18 05:00:19 2022] POST /identity/v3/users => generated 286 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:00:22.300016 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e95ec69-ee53-4f83-83a6-58e84298a007 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:00:22.300539 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e95ec69-ee53-4f83-83a6-58e84298a007 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:00:22.300657 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e95ec69-ee53-4f83-83a6-58e84298a007 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:00:22.301992 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 580/1139] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 05:00:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 05:00:22.307565 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19821a5a-eeb2-44fd-90e9-e4c3871fde74 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:00:22.307932 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19821a5a-eeb2-44fd-90e9-e4c3871fde74 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:00:22.308126 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19821a5a-eeb2-44fd-90e9-e4c3871fde74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:00:22.325234 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-19821a5a-eeb2-44fd-90e9-e4c3871fde74 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:00:22.326455 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-19821a5a-eeb2-44fd-90e9-e4c3871fde74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:00:22.391557 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 560/1140] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 05:00:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 18 05:00:22.415388 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e95ec69-ee53-4f83-83a6-58e84298a007 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:00:22.513995 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e95ec69-ee53-4f83-83a6-58e84298a007 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:00:22.541959 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:00:22.547485 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e86584-3c18-42ed-8e53-96be7b05f3ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:00:22.548256 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e86584-3c18-42ed-8e53-96be7b05f3ac None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:00:22.548537 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e86584-3c18-42ed-8e53-96be7b05f3ac None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:00:22.548810 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e86584-3c18-42ed-8e53-96be7b05f3ac None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:00:22.559411 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e86584-3c18-42ed-8e53-96be7b05f3ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 18 05:00:22.566853 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-07e86584-3c18-42ed-8e53-96be7b05f3ac None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 18 05:00:22.566853 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 05:00:22.566853 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 05:00:22.566853 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 05:00:22.566853 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 05:00:22.566853 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 05:00:22.566853 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 05:00:22.566853 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 05:00:22.566853 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 05:00:22.566853 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 05:00:22.566853 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 05:00:22.566853 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 05:00:22.566853 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 18 05:00:22.566853 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 18 05:00:22.566853 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 18 05:00:22.566853 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 18 05:00:22.568026 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 18 05:00:22.568026 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 18 05:00:22.568026 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 18 05:00:22.568026 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 18 05:00:22.568026 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 05:00:22.568026 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 05:00:22.568026 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 05:00:22.568026 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 05:00:22.568026 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 05:00:22.568026 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 05:00:22.568026 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 05:00:22.568026 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jun 18 05:00:22.568026 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 05:00:22.568026 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jun 18 05:00:22.568026 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 05:00:22.568026 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 05:00:22.569121 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 05:00:22.569121 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jun 18 05:00:22.569121 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 05:00:22.569121 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jun 18 05:00:22.569121 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 18 05:00:22.569121 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 18 05:00:22.569121 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 18 05:00:22.569121 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 18 05:00:22.569121 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 18 05:00:22.569121 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 18 05:00:22.569121 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 18 05:00:22.569121 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 05:00:22.569121 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 581/1141] 10.222.0.23 () {56 vars in 1096 bytes} [Sat Jun 18 05:00:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 05:00:22.574054 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19821a5a-eeb2-44fd-90e9-e4c3871fde74 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:00:22.619003 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:00:22.671296 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19821a5a-eeb2-44fd-90e9-e4c3871fde74 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:00:22.704649 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6fb133-4800-4357-aa1e-3a9290f378b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:00:22.705326 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6fb133-4800-4357-aa1e-3a9290f378b0 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:00:22.705650 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6fb133-4800-4357-aa1e-3a9290f378b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:00:22.705924 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6fb133-4800-4357-aa1e-3a9290f378b0 None admin] PATH_INFO: `/v3/roles` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:00:22.706661 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6fb133-4800-4357-aa1e-3a9290f378b0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 05:00:22.706973 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6fb133-4800-4357-aa1e-3a9290f378b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:00:22.708329 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6fb133-4800-4357-aa1e-3a9290f378b0 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:00:22.716845 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 561/1142] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Jun 18 05:00:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 05:00:22.722571 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e86584-3c18-42ed-8e53-96be7b05f3ac None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:00:22.823510 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e86584-3c18-42ed-8e53-96be7b05f3ac None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:00:22.856813 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c8789b-fb4b-4467-a6f6-70ccd3143236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:00:22.857689 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c8789b-fb4b-4467-a6f6-70ccd3143236 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:00:22.857968 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c8789b-fb4b-4467-a6f6-70ccd3143236 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:00:22.858234 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c8789b-fb4b-4467-a6f6-70ccd3143236 None admin] PATH_INFO: `/v3/users/1ebe6691136547f2858f5ed116a74343` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:00:22.871385 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c8789b-fb4b-4467-a6f6-70ccd3143236 None admin] RBAC: Authorizing `identity:get_user(user_id=1ebe6691136547f2858f5ed116a74343)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:00:22.872807 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c8789b-fb4b-4467-a6f6-70ccd3143236 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:00:22.874464 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 582/1143] 10.222.0.23 () {56 vars in 1177 bytes} [Sat Jun 18 05:00:22 2022] GET /identity/v3/users/1ebe6691136547f2858f5ed116a74343 => generated 286 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 05:00:22.880359 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6fb133-4800-4357-aa1e-3a9290f378b0 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:00:22.977616 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6fb133-4800-4357-aa1e-3a9290f378b0 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:00:22.984155 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:00:23.010257 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7159d3-762a-4aa9-a57c-caac2309e57a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:00:23.010873 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7159d3-762a-4aa9-a57c-caac2309e57a None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:00:23.011152 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7159d3-762a-4aa9-a57c-caac2309e57a None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:00:23.011422 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7159d3-762a-4aa9-a57c-caac2309e57a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:00:23.076081 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7159d3-762a-4aa9-a57c-caac2309e57a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:00:23.077608 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7159d3-762a-4aa9-a57c-caac2309e57a None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:00:23.172695 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 562/1144] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Jun 18 05:00:22 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 18 05:00:23.178717 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c8789b-fb4b-4467-a6f6-70ccd3143236 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:00:23.227333 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:00:23.276480 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c8789b-fb4b-4467-a6f6-70ccd3143236 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:00:23.309677 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146c0895-874a-4c4d-a064-1124db9adabb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:00:23.310523 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146c0895-874a-4c4d-a064-1124db9adabb None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:00:23.310810 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146c0895-874a-4c4d-a064-1124db9adabb None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:00:23.311087 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146c0895-874a-4c4d-a064-1124db9adabb None admin] PATH_INFO: `/v3/projects/61a2022fbc1c446ca144169f79d35014` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:00:23.312265 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146c0895-874a-4c4d-a064-1124db9adabb None admin] RBAC: Authorizing `identity:get_project(project_id=61a2022fbc1c446ca144169f79d35014)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:00:23.313731 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146c0895-874a-4c4d-a064-1124db9adabb None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:00:23.315413 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 583/1145] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Jun 18 05:00:23 2022] GET /identity/v3/projects/61a2022fbc1c446ca144169f79d35014 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 05:00:23.321597 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7159d3-762a-4aa9-a57c-caac2309e57a None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:00:23.329508 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:00:23.420357 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7159d3-762a-4aa9-a57c-caac2309e57a None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:00:23.453059 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a47ba0-1e4e-4a8a-a88d-c38001565f4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:00:23.453728 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a47ba0-1e4e-4a8a-a88d-c38001565f4a None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:00:23.454014 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a47ba0-1e4e-4a8a-a88d-c38001565f4a None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:00:23.454290 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a47ba0-1e4e-4a8a-a88d-c38001565f4a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:00:23.455108 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a47ba0-1e4e-4a8a-a88d-c38001565f4a None admin] RBAC: Adding query filter params (role.id=21c65f2c51bd4dd283d13507d2714467, scope.project.id=61a2022fbc1c446ca144169f79d35014, user.id=1ebe6691136547f2858f5ed116a74343) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 05:00:23.455424 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a47ba0-1e4e-4a8a-a88d-c38001565f4a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:00:23.456838 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a47ba0-1e4e-4a8a-a88d-c38001565f4a None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:00:23.462704 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 563/1146] 10.222.0.23 () {56 vars in 1374 bytes} [Sat Jun 18 05:00:23 2022] GET /identity/v3/role_assignments?user.id=1ebe6691136547f2858f5ed116a74343&scope.project.id=61a2022fbc1c446ca144169f79d35014&role.id=21c65f2c51bd4dd283d13507d2714467 => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 05:00:25.458068 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c16f3f-d572-4e4c-92bb-59325bf503bb None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:00:25.458533 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c16f3f-d572-4e4c-92bb-59325bf503bb None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:00:25.458644 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c16f3f-d572-4e4c-92bb-59325bf503bb None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:00:25.460465 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 584/1147] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 05:00:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 05:00:25.465654 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da421a1b-9383-4dd7-a29e-a9e7f3d31020 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:00:25.466085 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da421a1b-9383-4dd7-a29e-a9e7f3d31020 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:00:25.466215 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da421a1b-9383-4dd7-a29e-a9e7f3d31020 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:00:25.483084 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-da421a1b-9383-4dd7-a29e-a9e7f3d31020 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:00:25.484258 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-da421a1b-9383-4dd7-a29e-a9e7f3d31020 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:00:25.544943 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:00:25.549367 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 564/1148] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 05:00:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 18 05:00:25.572569 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c16f3f-d572-4e4c-92bb-59325bf503bb None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:00:25.671084 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c16f3f-d572-4e4c-92bb-59325bf503bb None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:00:25.687235 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:00:25.704490 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0c9171-2279-492f-aefd-37a1fee38946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:00:25.705254 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0c9171-2279-492f-aefd-37a1fee38946 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:00:25.705606 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0c9171-2279-492f-aefd-37a1fee38946 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:00:25.705876 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0c9171-2279-492f-aefd-37a1fee38946 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:00:25.715884 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0c9171-2279-492f-aefd-37a1fee38946 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 18 05:00:25.723843 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-5a0c9171-2279-492f-aefd-37a1fee38946 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 18 05:00:25.723843 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 05:00:25.723843 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 05:00:25.723843 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 05:00:25.723843 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 05:00:25.723843 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 05:00:25.723843 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 05:00:25.723843 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 05:00:25.723843 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 05:00:25.723843 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 05:00:25.723843 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 05:00:25.723843 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 05:00:25.723843 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 18 05:00:25.723843 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 18 05:00:25.723843 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 18 05:00:25.723843 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 18 05:00:25.725089 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 18 05:00:25.725089 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 18 05:00:25.725089 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 18 05:00:25.725089 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 18 05:00:25.725089 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 05:00:25.725089 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 05:00:25.725089 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 05:00:25.725089 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 05:00:25.725089 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 05:00:25.725089 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 05:00:25.725089 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 05:00:25.725089 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jun 18 05:00:25.725089 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 05:00:25.725089 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jun 18 05:00:25.725089 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 05:00:25.725089 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 05:00:25.726288 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 05:00:25.726288 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jun 18 05:00:25.726288 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 05:00:25.726288 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jun 18 05:00:25.726288 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 18 05:00:25.726288 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 18 05:00:25.726288 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 18 05:00:25.726288 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 18 05:00:25.726288 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 18 05:00:25.726288 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 18 05:00:25.726288 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 18 05:00:25.726288 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 05:00:25.726288 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 585/1149] 10.222.0.23 () {56 vars in 1096 bytes} [Sat Jun 18 05:00:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 05:00:25.731437 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da421a1b-9383-4dd7-a29e-a9e7f3d31020 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:00:25.827766 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da421a1b-9383-4dd7-a29e-a9e7f3d31020 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:00:25.860552 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751ecb02-041e-4cd0-98f0-caddb520b843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:00:25.861201 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751ecb02-041e-4cd0-98f0-caddb520b843 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:00:25.861515 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751ecb02-041e-4cd0-98f0-caddb520b843 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:00:25.861794 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751ecb02-041e-4cd0-98f0-caddb520b843 None admin] PATH_INFO: `/v3/roles` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:00:25.862527 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751ecb02-041e-4cd0-98f0-caddb520b843 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 05:00:25.862838 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751ecb02-041e-4cd0-98f0-caddb520b843 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:00:25.864234 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751ecb02-041e-4cd0-98f0-caddb520b843 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:00:25.872435 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 565/1150] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Jun 18 05:00:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 05:00:25.878046 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0c9171-2279-492f-aefd-37a1fee38946 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:00:25.978645 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0c9171-2279-492f-aefd-37a1fee38946 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:00:26.010806 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61303922-c232-43c2-8cd0-877eb82b6764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:00:26.011647 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61303922-c232-43c2-8cd0-877eb82b6764 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:00:26.011923 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61303922-c232-43c2-8cd0-877eb82b6764 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:00:26.012189 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61303922-c232-43c2-8cd0-877eb82b6764 None admin] PATH_INFO: `/v3/users/1ebe6691136547f2858f5ed116a74343` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:00:26.013353 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61303922-c232-43c2-8cd0-877eb82b6764 None admin] RBAC: Authorizing `identity:get_user(user_id=1ebe6691136547f2858f5ed116a74343)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:00:26.014811 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61303922-c232-43c2-8cd0-877eb82b6764 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:00:26.016400 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 586/1151] 10.222.0.23 () {56 vars in 1177 bytes} [Sat Jun 18 05:00:25 2022] GET /identity/v3/users/1ebe6691136547f2858f5ed116a74343 => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 05:00:26.022056 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751ecb02-041e-4cd0-98f0-caddb520b843 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:00:26.082695 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:00:26.120776 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751ecb02-041e-4cd0-98f0-caddb520b843 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:00:26.153575 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f63a121-428d-4409-944d-63ac45908ff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:00:26.154407 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f63a121-428d-4409-944d-63ac45908ff4 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:00:26.154687 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f63a121-428d-4409-944d-63ac45908ff4 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:00:26.154961 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f63a121-428d-4409-944d-63ac45908ff4 None admin] PATH_INFO: `/v3/projects/61a2022fbc1c446ca144169f79d35014` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:00:26.156111 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f63a121-428d-4409-944d-63ac45908ff4 None admin] RBAC: Authorizing `identity:get_project(project_id=61a2022fbc1c446ca144169f79d35014)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:00:26.157552 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f63a121-428d-4409-944d-63ac45908ff4 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:00:26.159199 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 566/1152] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Jun 18 05:00:26 2022] GET /identity/v3/projects/61a2022fbc1c446ca144169f79d35014 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 05:00:26.164968 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61303922-c232-43c2-8cd0-877eb82b6764 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:00:26.210135 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:00:26.263738 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61303922-c232-43c2-8cd0-877eb82b6764 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:00:26.295636 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a8937c-8ecd-4a5c-b5bb-0af7c5324124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:00:26.296511 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a8937c-8ecd-4a5c-b5bb-0af7c5324124 None admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:00:26.296792 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a8937c-8ecd-4a5c-b5bb-0af7c5324124 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:00:26.297069 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a8937c-8ecd-4a5c-b5bb-0af7c5324124 None admin] PATH_INFO: `/v3/projects/61a2022fbc1c446ca144169f79d35014/users/1ebe6691136547f2858f5ed116a74343/roles/21c65f2c51bd4dd283d13507d2714467` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:00:26.299058 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a8937c-8ecd-4a5c-b5bb-0af7c5324124 None admin] RBAC: Authorizing `identity:create_grant(project_id=61a2022fbc1c446ca144169f79d35014, user_id=1ebe6691136547f2858f5ed116a74343, role_id=21c65f2c51bd4dd283d13507d2714467)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:00:26.300487 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a8937c-8ecd-4a5c-b5bb-0af7c5324124 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:00:26.385779 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 587/1153] 10.222.0.23 () {58 vars in 1439 bytes} [Sat Jun 18 05:00:26 2022] PUT /identity/v3/projects/61a2022fbc1c446ca144169f79d35014/users/1ebe6691136547f2858f5ed116a74343/roles/21c65f2c51bd4dd283d13507d2714467 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:00:28.391672 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9081d8d-076f-4db3-8ecb-e829500b91d1 None None] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:00:28.392065 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9081d8d-076f-4db3-8ecb-e829500b91d1 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:00:28.392269 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9081d8d-076f-4db3-8ecb-e829500b91d1 None None] PATH_INFO: `/` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:00:28.393680 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 567/1154] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 05:00:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 05:00:28.399526 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7624d0-c7a0-456d-b329-83fab9bbb017 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:00:28.399896 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7624d0-c7a0-456d-b329-83fab9bbb017 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:00:28.400082 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7624d0-c7a0-456d-b329-83fab9bbb017 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:00:28.416493 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-2b7624d0-c7a0-456d-b329-83fab9bbb017 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:00:28.417724 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-2b7624d0-c7a0-456d-b329-83fab9bbb017 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:00:28.482040 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 588/1155] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 05:00:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 18 05:00:28.505763 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9081d8d-076f-4db3-8ecb-e829500b91d1 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:00:28.605172 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9081d8d-076f-4db3-8ecb-e829500b91d1 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:00:28.633134 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:00:28.638665 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e032fc-bd3e-453b-be90-ed74ab4b19d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:00:28.639435 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e032fc-bd3e-453b-be90-ed74ab4b19d1 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:00:28.639720 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e032fc-bd3e-453b-be90-ed74ab4b19d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:00:28.640043 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e032fc-bd3e-453b-be90-ed74ab4b19d1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:00:28.650861 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e032fc-bd3e-453b-be90-ed74ab4b19d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 18 05:00:28.658344 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application [None req-b8e032fc-bd3e-453b-be90-ed74ab4b19d1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 18 05:00:28.658344 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 05:00:28.658344 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 05:00:28.658344 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 05:00:28.658344 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 05:00:28.658344 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 05:00:28.658344 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 05:00:28.658344 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 05:00:28.658344 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 05:00:28.658344 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 05:00:28.658344 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 05:00:28.658344 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 05:00:28.658344 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 18 05:00:28.658344 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 18 05:00:28.658344 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 18 05:00:28.658344 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 18 05:00:28.659405 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 18 05:00:28.659405 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 18 05:00:28.659405 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 18 05:00:28.659405 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 18 05:00:28.659405 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 05:00:28.659405 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 05:00:28.659405 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 05:00:28.659405 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 05:00:28.659405 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 05:00:28.659405 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 05:00:28.659405 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 05:00:28.659405 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application with Lock( Jun 18 05:00:28.659405 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 05:00:28.659405 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._enter() Jun 18 05:00:28.659405 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 05:00:28.659405 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 05:00:28.660674 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 05:00:28.660674 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.creator() Jun 18 05:00:28.660674 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 05:00:28.660674 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application created_value = creator( Jun 18 05:00:28.660674 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 18 05:00:28.660674 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 18 05:00:28.660674 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 18 05:00:28.660674 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 18 05:00:28.660674 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 18 05:00:28.660674 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 18 05:00:28.660674 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 18 05:00:28.660674 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 05:00:28.660674 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 568/1156] 10.222.0.23 () {56 vars in 1096 bytes} [Sat Jun 18 05:00:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 05:00:28.666148 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7624d0-c7a0-456d-b329-83fab9bbb017 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:00:28.698175 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:00:28.762291 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7624d0-c7a0-456d-b329-83fab9bbb017 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:00:28.795759 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f28ee82-2724-4c7c-967a-bae1d37bb154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:00:28.796417 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f28ee82-2724-4c7c-967a-bae1d37bb154 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:00:28.796703 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f28ee82-2724-4c7c-967a-bae1d37bb154 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:00:28.797009 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f28ee82-2724-4c7c-967a-bae1d37bb154 None admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:00:28.797793 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f28ee82-2724-4c7c-967a-bae1d37bb154 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 05:00:28.798283 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f28ee82-2724-4c7c-967a-bae1d37bb154 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:00:28.800532 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f28ee82-2724-4c7c-967a-bae1d37bb154 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:00:28.808594 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 589/1157] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Jun 18 05:00:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 05:00:28.814368 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e032fc-bd3e-453b-be90-ed74ab4b19d1 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:00:28.914565 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e032fc-bd3e-453b-be90-ed74ab4b19d1 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:00:28.947742 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d76ac3-c2dd-4a44-a837-ba7d9a73f0fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:00:28.948573 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d76ac3-c2dd-4a44-a837-ba7d9a73f0fd None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:00:28.948853 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d76ac3-c2dd-4a44-a837-ba7d9a73f0fd None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:00:28.949124 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d76ac3-c2dd-4a44-a837-ba7d9a73f0fd None admin] PATH_INFO: `/v3/users/1ebe6691136547f2858f5ed116a74343` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:00:28.950385 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d76ac3-c2dd-4a44-a837-ba7d9a73f0fd None admin] RBAC: Authorizing `identity:get_user(user_id=1ebe6691136547f2858f5ed116a74343)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:00:28.951795 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d76ac3-c2dd-4a44-a837-ba7d9a73f0fd None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:00:28.953370 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 569/1158] 10.222.0.23 () {56 vars in 1177 bytes} [Sat Jun 18 05:00:28 2022] GET /identity/v3/users/1ebe6691136547f2858f5ed116a74343 => generated 286 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 05:00:28.959109 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f28ee82-2724-4c7c-967a-bae1d37bb154 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:00:29.052132 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:00:29.058155 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f28ee82-2724-4c7c-967a-bae1d37bb154 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:00:29.092116 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cd754f-d22d-4724-891c-6810c4cf651c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:00:29.092749 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cd754f-d22d-4724-891c-6810c4cf651c None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:00:29.093035 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cd754f-d22d-4724-891c-6810c4cf651c None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:00:29.093316 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cd754f-d22d-4724-891c-6810c4cf651c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:00:29.158431 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cd754f-d22d-4724-891c-6810c4cf651c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:00:29.159876 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cd754f-d22d-4724-891c-6810c4cf651c None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:00:29.248763 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:00:29.255039 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 590/1159] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Jun 18 05:00:28 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 18 05:00:29.261585 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d76ac3-c2dd-4a44-a837-ba7d9a73f0fd None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:00:29.318381 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:00:29.357449 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d76ac3-c2dd-4a44-a837-ba7d9a73f0fd None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:00:29.392488 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc6c1db-f49e-44cb-9bfd-8124f8fb5836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:00:29.392953 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc6c1db-f49e-44cb-9bfd-8124f8fb5836 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:00:29.393090 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc6c1db-f49e-44cb-9bfd-8124f8fb5836 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:00:29.393363 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc6c1db-f49e-44cb-9bfd-8124f8fb5836 None admin] PATH_INFO: `/v3/projects/61a2022fbc1c446ca144169f79d35014` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:00:29.394571 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc6c1db-f49e-44cb-9bfd-8124f8fb5836 None admin] RBAC: Authorizing `identity:get_project(project_id=61a2022fbc1c446ca144169f79d35014)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:00:29.395990 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc6c1db-f49e-44cb-9bfd-8124f8fb5836 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:00:29.397803 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 570/1160] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Jun 18 05:00:29 2022] GET /identity/v3/projects/61a2022fbc1c446ca144169f79d35014 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 05:00:29.403988 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cd754f-d22d-4724-891c-6810c4cf651c None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:00:29.501094 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cd754f-d22d-4724-891c-6810c4cf651c None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:00:29.534210 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f30352-ff73-411b-8b6b-390705589abf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:00:29.534842 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f30352-ff73-411b-8b6b-390705589abf None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:00:29.535127 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f30352-ff73-411b-8b6b-390705589abf None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:00:29.535403 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f30352-ff73-411b-8b6b-390705589abf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:00:29.536218 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f30352-ff73-411b-8b6b-390705589abf None admin] RBAC: Adding query filter params (role.id=21c65f2c51bd4dd283d13507d2714467, scope.project.id=61a2022fbc1c446ca144169f79d35014, user.id=1ebe6691136547f2858f5ed116a74343) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 05:00:29.536534 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f30352-ff73-411b-8b6b-390705589abf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:00:29.538170 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f30352-ff73-411b-8b6b-390705589abf None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:00:29.543899 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 591/1161] 10.222.0.23 () {56 vars in 1374 bytes} [Sat Jun 18 05:00:29 2022] GET /identity/v3/role_assignments?user.id=1ebe6691136547f2858f5ed116a74343&scope.project.id=61a2022fbc1c446ca144169f79d35014&role.id=21c65f2c51bd4dd283d13507d2714467 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 05:00:31.589373 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be7d84b-b7bb-4c4d-a7ad-7aed70c8f14c None None] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:00:31.589923 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be7d84b-b7bb-4c4d-a7ad-7aed70c8f14c None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:00:31.589923 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be7d84b-b7bb-4c4d-a7ad-7aed70c8f14c None None] PATH_INFO: `/` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:00:31.591292 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 571/1162] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 05:00:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 05:00:31.596357 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1afc47-26ba-4466-988d-04fb4f8ed21c None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:00:31.596723 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1afc47-26ba-4466-988d-04fb4f8ed21c None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:00:31.596918 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1afc47-26ba-4466-988d-04fb4f8ed21c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:00:31.614153 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-9a1afc47-26ba-4466-988d-04fb4f8ed21c None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:00:31.615370 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-9a1afc47-26ba-4466-988d-04fb4f8ed21c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:00:31.670184 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:00:31.680921 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 592/1163] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 05:00:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 18 05:00:31.703595 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be7d84b-b7bb-4c4d-a7ad-7aed70c8f14c None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:00:31.805033 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be7d84b-b7bb-4c4d-a7ad-7aed70c8f14c None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:00:31.830178 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:00:31.839050 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb1e20a-83b3-4cde-acd6-6cddb93169b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:00:31.839757 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb1e20a-83b3-4cde-acd6-6cddb93169b0 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:00:31.840040 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb1e20a-83b3-4cde-acd6-6cddb93169b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:00:31.840312 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb1e20a-83b3-4cde-acd6-6cddb93169b0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:00:31.905871 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb1e20a-83b3-4cde-acd6-6cddb93169b0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:00:31.907297 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb1e20a-83b3-4cde-acd6-6cddb93169b0 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:00:32.002299 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 572/1164] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Jun 18 05:00:31 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 18 05:00:32.009573 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1afc47-26ba-4466-988d-04fb4f8ed21c None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:00:32.105526 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1afc47-26ba-4466-988d-04fb4f8ed21c None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:00:32.138830 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e84c279-7f45-46c4-bca7-a6a1020a6365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:00:32.139611 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e84c279-7f45-46c4-bca7-a6a1020a6365 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:00:32.139894 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e84c279-7f45-46c4-bca7-a6a1020a6365 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:00:32.140168 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e84c279-7f45-46c4-bca7-a6a1020a6365 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:00:32.147289 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e84c279-7f45-46c4-bca7-a6a1020a6365 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:00:32.148740 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e84c279-7f45-46c4-bca7-a6a1020a6365 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:00:32.162523 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-5e84c279-7f45-46c4-bca7-a6a1020a6365 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 18 05:00:32.162523 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 05:00:32.162523 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jun 18 05:00:32.162523 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jun 18 05:00:32.162523 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 18 05:00:32.162523 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 18 05:00:32.162523 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 18 05:00:32.162523 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 18 05:00:32.162523 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jun 18 05:00:32.162523 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 05:00:32.162523 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 18 05:00:32.162523 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 05:00:32.162523 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 05:00:32.162523 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 05:00:32.162523 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 05:00:32.162523 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 05:00:32.162523 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 05:00:32.163788 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 05:00:32.163788 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 05:00:32.163788 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 05:00:32.163788 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 05:00:32.163788 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 05:00:32.163788 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 05:00:32.163788 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 18 05:00:32.163788 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 18 05:00:32.163788 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 18 05:00:32.163788 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 18 05:00:32.163788 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 05:00:32.163788 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 05:00:32.163788 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 05:00:32.163788 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 05:00:32.163788 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 05:00:32.163788 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 05:00:32.164659 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 05:00:32.164659 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jun 18 05:00:32.164659 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 05:00:32.164659 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jun 18 05:00:32.164659 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 05:00:32.164659 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 05:00:32.164659 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 05:00:32.164659 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jun 18 05:00:32.164659 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 05:00:32.164659 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jun 18 05:00:32.164659 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jun 18 05:00:32.164659 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 18 05:00:32.164659 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 18 05:00:32.164659 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 05:00:32.164659 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 593/1165] 10.222.0.23 () {56 vars in 1117 bytes} [Sat Jun 18 05:00:32 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 05:00:32.169855 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb1e20a-83b3-4cde-acd6-6cddb93169b0 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:00:32.196781 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:00:32.267945 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb1e20a-83b3-4cde-acd6-6cddb93169b0 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:00:32.300315 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d546aca-d9c3-4d96-9352-d64cf30d5fec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:00:32.300958 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d546aca-d9c3-4d96-9352-d64cf30d5fec None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:00:32.301244 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d546aca-d9c3-4d96-9352-d64cf30d5fec None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:00:32.301554 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d546aca-d9c3-4d96-9352-d64cf30d5fec None admin] PATH_INFO: `/v3/domains` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:00:32.302271 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d546aca-d9c3-4d96-9352-d64cf30d5fec None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 05:00:32.302584 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d546aca-d9c3-4d96-9352-d64cf30d5fec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:00:32.304580 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d546aca-d9c3-4d96-9352-d64cf30d5fec None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:00:32.311588 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 573/1166] 10.222.0.23 () {56 vars in 1115 bytes} [Sat Jun 18 05:00:32 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 05:00:34.343997 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cfffde-bdf2-44b4-92cc-58223822e5c8 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:00:34.344525 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cfffde-bdf2-44b4-92cc-58223822e5c8 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:00:34.344525 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cfffde-bdf2-44b4-92cc-58223822e5c8 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:00:34.345971 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 594/1167] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 05:00:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 05:00:34.351087 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9510ff75-0c31-449f-ac43-265e276f4f2f None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:00:34.351461 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9510ff75-0c31-449f-ac43-265e276f4f2f None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:00:34.351651 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9510ff75-0c31-449f-ac43-265e276f4f2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:00:34.368368 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-9510ff75-0c31-449f-ac43-265e276f4f2f None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:00:34.369602 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-9510ff75-0c31-449f-ac43-265e276f4f2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:00:34.436333 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 574/1168] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 05:00:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 18 05:00:34.459677 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cfffde-bdf2-44b4-92cc-58223822e5c8 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:00:34.509886 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:00:34.560648 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cfffde-bdf2-44b4-92cc-58223822e5c8 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:00:34.593850 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec37b17d-fcf3-4c87-99ed-d113ccb2a98a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:00:34.594548 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec37b17d-fcf3-4c87-99ed-d113ccb2a98a None admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:00:34.594778 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec37b17d-fcf3-4c87-99ed-d113ccb2a98a None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:00:34.595052 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec37b17d-fcf3-4c87-99ed-d113ccb2a98a None admin] PATH_INFO: `/v3/domains` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:00:34.595796 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec37b17d-fcf3-4c87-99ed-d113ccb2a98a None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:00:34.597162 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec37b17d-fcf3-4c87-99ed-d113ccb2a98a None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:00:34.656752 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 595/1169] 10.222.0.23 () {60 vars in 1138 bytes} [Sat Jun 18 05:00:34 2022] POST /identity/v3/domains => generated 267 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:00:36.697544 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b407d837-7d79-4ed5-838c-ae10a03ae62b None None] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:00:36.698086 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b407d837-7d79-4ed5-838c-ae10a03ae62b None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:00:36.698190 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b407d837-7d79-4ed5-838c-ae10a03ae62b None None] PATH_INFO: `/` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:00:36.699487 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 575/1170] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 05:00:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 05:00:36.705417 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea415c7d-1f3b-43fa-8d55-23a2984dae67 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:00:36.705846 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea415c7d-1f3b-43fa-8d55-23a2984dae67 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:00:36.705989 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea415c7d-1f3b-43fa-8d55-23a2984dae67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:00:36.724560 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-ea415c7d-1f3b-43fa-8d55-23a2984dae67 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:00:36.726418 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-ea415c7d-1f3b-43fa-8d55-23a2984dae67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:00:36.808080 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 596/1171] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 05:00:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 18 05:00:36.831390 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b407d837-7d79-4ed5-838c-ae10a03ae62b None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:00:36.857226 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:00:36.935477 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b407d837-7d79-4ed5-838c-ae10a03ae62b None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:00:36.968393 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb292f4-9162-4ef9-80ee-b64f785e31ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:00:36.969175 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb292f4-9162-4ef9-80ee-b64f785e31ca None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:00:36.969487 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb292f4-9162-4ef9-80ee-b64f785e31ca None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:00:36.969766 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb292f4-9162-4ef9-80ee-b64f785e31ca None admin] PATH_INFO: `/v3/domains/d013dd4d8e2d48faaf17835aa96120cb` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:00:36.980666 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb292f4-9162-4ef9-80ee-b64f785e31ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=d013dd4d8e2d48faaf17835aa96120cb)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:00:36.982128 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb292f4-9162-4ef9-80ee-b64f785e31ca None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:00:36.983741 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 576/1172] 10.222.0.23 () {56 vars in 1183 bytes} [Sat Jun 18 05:00:36 2022] GET /identity/v3/domains/d013dd4d8e2d48faaf17835aa96120cb => generated 267 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 05:00:36.989555 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea415c7d-1f3b-43fa-8d55-23a2984dae67 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:00:37.030414 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:00:37.088356 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea415c7d-1f3b-43fa-8d55-23a2984dae67 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:00:37.121921 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64699dab-c574-4a6e-8ea3-c76e3de76851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:00:37.122563 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64699dab-c574-4a6e-8ea3-c76e3de76851 None admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:00:37.122844 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64699dab-c574-4a6e-8ea3-c76e3de76851 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:00:37.123119 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64699dab-c574-4a6e-8ea3-c76e3de76851 None admin] PATH_INFO: `/v3/projects` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:00:37.123825 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64699dab-c574-4a6e-8ea3-c76e3de76851 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:00:37.125261 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64699dab-c574-4a6e-8ea3-c76e3de76851 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:00:37.191597 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 597/1173] 10.222.0.23 () {60 vars in 1141 bytes} [Sat Jun 18 05:00:36 2022] POST /identity/v3/projects => generated 361 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 18 05:00:39.238871 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee788c7-1d11-415d-8907-7a9c392c9018 None None] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:00:39.239458 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee788c7-1d11-415d-8907-7a9c392c9018 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:00:39.239458 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee788c7-1d11-415d-8907-7a9c392c9018 None None] PATH_INFO: `/` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:00:39.240849 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 577/1174] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 05:00:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 05:00:39.246388 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fa996c-c411-47cf-ba26-3dbcc7d43892 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:00:39.246781 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fa996c-c411-47cf-ba26-3dbcc7d43892 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:00:39.246957 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fa996c-c411-47cf-ba26-3dbcc7d43892 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:00:39.263272 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-f6fa996c-c411-47cf-ba26-3dbcc7d43892 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:00:39.264491 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-f6fa996c-c411-47cf-ba26-3dbcc7d43892 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:00:39.330148 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 598/1175] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 05:00:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 18 05:00:39.355829 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee788c7-1d11-415d-8907-7a9c392c9018 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:00:39.447310 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:00:39.467358 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee788c7-1d11-415d-8907-7a9c392c9018 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:00:39.501083 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15deff8c-d93b-47d8-993e-d2bb9725aea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:00:39.502031 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15deff8c-d93b-47d8-993e-d2bb9725aea9 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:00:39.502312 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15deff8c-d93b-47d8-993e-d2bb9725aea9 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:00:39.502582 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15deff8c-d93b-47d8-993e-d2bb9725aea9 None admin] PATH_INFO: `/v3/domains/d013dd4d8e2d48faaf17835aa96120cb` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:00:39.503750 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15deff8c-d93b-47d8-993e-d2bb9725aea9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=d013dd4d8e2d48faaf17835aa96120cb)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:00:39.505165 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15deff8c-d93b-47d8-993e-d2bb9725aea9 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:00:39.507419 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 578/1176] 10.222.0.23 () {56 vars in 1183 bytes} [Sat Jun 18 05:00:39 2022] GET /identity/v3/domains/d013dd4d8e2d48faaf17835aa96120cb => generated 267 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 05:00:39.513166 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fa996c-c411-47cf-ba26-3dbcc7d43892 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:00:39.536598 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:00:39.609729 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fa996c-c411-47cf-ba26-3dbcc7d43892 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:00:39.642837 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad7636a-94d7-4b39-8a22-d907fea3a151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:00:39.643490 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad7636a-94d7-4b39-8a22-d907fea3a151 None admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:00:39.643776 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad7636a-94d7-4b39-8a22-d907fea3a151 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:00:39.644047 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad7636a-94d7-4b39-8a22-d907fea3a151 None admin] PATH_INFO: `/v3/users` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:00:39.644744 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad7636a-94d7-4b39-8a22-d907fea3a151 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:00:39.646195 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad7636a-94d7-4b39-8a22-d907fea3a151 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:00:39.734131 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 599/1177] 10.222.0.23 () {60 vars in 1132 bytes} [Sat Jun 18 05:00:39 2022] POST /identity/v3/users => generated 311 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:00:41.789365 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2bfbe0-3d5e-44ba-8c60-87773781d05c None None] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:00:41.789926 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2bfbe0-3d5e-44ba-8c60-87773781d05c None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:00:41.789926 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2bfbe0-3d5e-44ba-8c60-87773781d05c None None] PATH_INFO: `/` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:00:41.791280 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 579/1178] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 05:00:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 05:00:41.796484 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bca7de2-f7c7-4752-a198-610863a43509 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:00:41.796926 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bca7de2-f7c7-4752-a198-610863a43509 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:00:41.797027 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bca7de2-f7c7-4752-a198-610863a43509 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:00:41.814309 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-0bca7de2-f7c7-4752-a198-610863a43509 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:00:41.815506 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-0bca7de2-f7c7-4752-a198-610863a43509 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:00:41.880496 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 600/1179] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 05:00:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 18 05:00:41.904159 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2bfbe0-3d5e-44ba-8c60-87773781d05c None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:00:42.002761 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2bfbe0-3d5e-44ba-8c60-87773781d05c None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:00:42.035308 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f085fc0-c9bd-4475-8847-ba0c831f4fe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:00:42.036067 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f085fc0-c9bd-4475-8847-ba0c831f4fe6 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:00:42.036348 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f085fc0-c9bd-4475-8847-ba0c831f4fe6 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:00:42.036619 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f085fc0-c9bd-4475-8847-ba0c831f4fe6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:00:42.052614 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f085fc0-c9bd-4475-8847-ba0c831f4fe6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 18 05:00:42.063322 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application [None req-1f085fc0-c9bd-4475-8847-ba0c831f4fe6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 18 05:00:42.063322 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 05:00:42.063322 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 05:00:42.063322 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 05:00:42.063322 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 05:00:42.063322 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 05:00:42.063322 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 05:00:42.063322 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 05:00:42.063322 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 05:00:42.063322 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 05:00:42.063322 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 05:00:42.063322 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 05:00:42.063322 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 18 05:00:42.063322 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 18 05:00:42.063322 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 18 05:00:42.063322 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 18 05:00:42.064323 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 18 05:00:42.064323 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 18 05:00:42.064323 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 18 05:00:42.064323 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 18 05:00:42.064323 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 05:00:42.064323 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 05:00:42.064323 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 05:00:42.064323 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 05:00:42.064323 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 05:00:42.064323 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 05:00:42.064323 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 05:00:42.064323 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application with Lock( Jun 18 05:00:42.064323 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 05:00:42.064323 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._enter() Jun 18 05:00:42.064323 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 05:00:42.064323 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 05:00:42.065224 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 05:00:42.065224 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.creator() Jun 18 05:00:42.065224 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 05:00:42.065224 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application created_value = creator( Jun 18 05:00:42.065224 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 18 05:00:42.065224 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 18 05:00:42.065224 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 18 05:00:42.065224 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 18 05:00:42.065224 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 18 05:00:42.065224 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 18 05:00:42.065224 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 18 05:00:42.065224 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 05:00:42.065947 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 580/1180] 10.222.0.23 () {56 vars in 1096 bytes} [Sat Jun 18 05:00:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 05:00:42.071465 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bca7de2-f7c7-4752-a198-610863a43509 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:00:42.081442 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:00:42.170326 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bca7de2-f7c7-4752-a198-610863a43509 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:00:42.203077 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c62be1-5907-4789-9158-1354b17d7c25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:00:42.203719 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c62be1-5907-4789-9158-1354b17d7c25 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:00:42.204000 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c62be1-5907-4789-9158-1354b17d7c25 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:00:42.204271 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c62be1-5907-4789-9158-1354b17d7c25 None admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:00:42.205002 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c62be1-5907-4789-9158-1354b17d7c25 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 05:00:42.205307 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c62be1-5907-4789-9158-1354b17d7c25 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:00:42.206693 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c62be1-5907-4789-9158-1354b17d7c25 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:00:42.215566 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 601/1181] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Jun 18 05:00:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 05:00:42.221209 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f085fc0-c9bd-4475-8847-ba0c831f4fe6 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:00:42.232927 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:00:42.319892 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f085fc0-c9bd-4475-8847-ba0c831f4fe6 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:00:42.352652 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bdddef-2ea5-44f2-9f0b-303beba6c586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:00:42.353533 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bdddef-2ea5-44f2-9f0b-303beba6c586 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:00:42.353821 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bdddef-2ea5-44f2-9f0b-303beba6c586 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:00:42.354094 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bdddef-2ea5-44f2-9f0b-303beba6c586 None admin] PATH_INFO: `/v3/users/4fa34bd49f1e46a699e051a9d9ff6a72` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:00:42.367547 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bdddef-2ea5-44f2-9f0b-303beba6c586 None admin] RBAC: Authorizing `identity:get_user(user_id=4fa34bd49f1e46a699e051a9d9ff6a72)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:00:42.369238 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bdddef-2ea5-44f2-9f0b-303beba6c586 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:00:42.370912 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 581/1182] 10.222.0.23 () {56 vars in 1177 bytes} [Sat Jun 18 05:00:42 2022] GET /identity/v3/users/4fa34bd49f1e46a699e051a9d9ff6a72 => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 05:00:42.376656 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c62be1-5907-4789-9158-1354b17d7c25 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:00:42.445855 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:00:42.476239 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c62be1-5907-4789-9158-1354b17d7c25 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:00:42.509636 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ba672e-85da-4e8a-a414-1f895a5e3f98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:00:42.510266 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ba672e-85da-4e8a-a414-1f895a5e3f98 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:00:42.510550 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ba672e-85da-4e8a-a414-1f895a5e3f98 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:00:42.510824 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ba672e-85da-4e8a-a414-1f895a5e3f98 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:00:42.575775 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ba672e-85da-4e8a-a414-1f895a5e3f98 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:00:42.577159 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ba672e-85da-4e8a-a414-1f895a5e3f98 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:00:42.645232 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:00:42.676179 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 602/1183] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Jun 18 05:00:42 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 18 05:00:42.682974 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bdddef-2ea5-44f2-9f0b-303beba6c586 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:00:42.749124 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:00:42.783906 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bdddef-2ea5-44f2-9f0b-303beba6c586 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:00:42.817730 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a30d3a8-cb4d-4b1d-8ec3-8b8a808e013d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:00:42.819157 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a30d3a8-cb4d-4b1d-8ec3-8b8a808e013d None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:00:42.819439 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a30d3a8-cb4d-4b1d-8ec3-8b8a808e013d None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:00:42.819710 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a30d3a8-cb4d-4b1d-8ec3-8b8a808e013d None admin] PATH_INFO: `/v3/projects/f58c08e7bed64e45b1eb401b297e97d7` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:00:42.820891 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a30d3a8-cb4d-4b1d-8ec3-8b8a808e013d None admin] RBAC: Authorizing `identity:get_project(project_id=f58c08e7bed64e45b1eb401b297e97d7)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:00:42.822345 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a30d3a8-cb4d-4b1d-8ec3-8b8a808e013d None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:00:42.824019 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 582/1184] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Jun 18 05:00:42 2022] GET /identity/v3/projects/f58c08e7bed64e45b1eb401b297e97d7 => generated 361 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 05:00:42.830237 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ba672e-85da-4e8a-a414-1f895a5e3f98 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:00:42.925150 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ba672e-85da-4e8a-a414-1f895a5e3f98 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:00:42.958660 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e168a20-5f58-4620-a53f-536001d844ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:00:42.959432 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e168a20-5f58-4620-a53f-536001d844ea None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:00:42.959609 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e168a20-5f58-4620-a53f-536001d844ea None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:00:42.959886 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e168a20-5f58-4620-a53f-536001d844ea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:00:42.960698 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e168a20-5f58-4620-a53f-536001d844ea None admin] RBAC: Adding query filter params (role.id=21c65f2c51bd4dd283d13507d2714467, scope.project.id=f58c08e7bed64e45b1eb401b297e97d7, user.id=4fa34bd49f1e46a699e051a9d9ff6a72) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 05:00:42.961010 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e168a20-5f58-4620-a53f-536001d844ea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:00:42.962481 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e168a20-5f58-4620-a53f-536001d844ea None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:00:42.967761 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 603/1185] 10.222.0.23 () {56 vars in 1374 bytes} [Sat Jun 18 05:00:42 2022] GET /identity/v3/role_assignments?user.id=4fa34bd49f1e46a699e051a9d9ff6a72&scope.project.id=f58c08e7bed64e45b1eb401b297e97d7&role.id=21c65f2c51bd4dd283d13507d2714467 => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 05:00:44.948738 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e9f5dc-c8af-42cb-933c-81d44f6bfc25 None None] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:00:44.949284 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e9f5dc-c8af-42cb-933c-81d44f6bfc25 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:00:44.949284 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e9f5dc-c8af-42cb-933c-81d44f6bfc25 None None] PATH_INFO: `/` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:00:44.950669 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 583/1186] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 05:00:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 05:00:44.955872 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63be7578-c5ed-4b11-bd59-82c6efef93de None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:00:44.956266 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63be7578-c5ed-4b11-bd59-82c6efef93de None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:00:44.956429 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63be7578-c5ed-4b11-bd59-82c6efef93de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:00:44.973304 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-63be7578-c5ed-4b11-bd59-82c6efef93de None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:00:44.974522 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-63be7578-c5ed-4b11-bd59-82c6efef93de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:00:45.000984 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:00:45.040569 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 604/1187] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 05:00:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 18 05:00:45.062827 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e9f5dc-c8af-42cb-933c-81d44f6bfc25 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:00:45.163474 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e9f5dc-c8af-42cb-933c-81d44f6bfc25 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:00:45.195698 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040dbcd2-fd9d-4995-98d4-b8d84b785650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:00:45.196463 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040dbcd2-fd9d-4995-98d4-b8d84b785650 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:00:45.196745 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040dbcd2-fd9d-4995-98d4-b8d84b785650 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:00:45.197014 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040dbcd2-fd9d-4995-98d4-b8d84b785650 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:00:45.198915 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:00:45.207341 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040dbcd2-fd9d-4995-98d4-b8d84b785650 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 18 05:00:45.215432 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application [None req-040dbcd2-fd9d-4995-98d4-b8d84b785650 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 18 05:00:45.215432 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 05:00:45.215432 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 05:00:45.215432 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 05:00:45.215432 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 05:00:45.215432 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 05:00:45.215432 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 05:00:45.215432 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 05:00:45.215432 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 05:00:45.215432 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 05:00:45.215432 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 05:00:45.215432 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 05:00:45.215432 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 18 05:00:45.215432 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 18 05:00:45.215432 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 18 05:00:45.215432 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 18 05:00:45.216556 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 18 05:00:45.216556 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 18 05:00:45.216556 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 18 05:00:45.216556 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 18 05:00:45.216556 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 05:00:45.216556 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 05:00:45.216556 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 05:00:45.216556 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 05:00:45.216556 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 05:00:45.216556 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 05:00:45.216556 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 05:00:45.216556 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application with Lock( Jun 18 05:00:45.216556 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 05:00:45.216556 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._enter() Jun 18 05:00:45.216556 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 05:00:45.216556 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 05:00:45.217622 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 05:00:45.217622 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.creator() Jun 18 05:00:45.217622 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 05:00:45.217622 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application created_value = creator( Jun 18 05:00:45.217622 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 18 05:00:45.217622 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 18 05:00:45.217622 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 18 05:00:45.217622 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 18 05:00:45.217622 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 18 05:00:45.217622 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 18 05:00:45.217622 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 18 05:00:45.217622 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 05:00:45.217622 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 584/1188] 10.222.0.23 () {56 vars in 1096 bytes} [Sat Jun 18 05:00:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 05:00:45.222785 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63be7578-c5ed-4b11-bd59-82c6efef93de None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:00:45.318318 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63be7578-c5ed-4b11-bd59-82c6efef93de None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:00:45.351702 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ef7dc3-5669-43da-8923-0ac221edd518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:00:45.352347 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ef7dc3-5669-43da-8923-0ac221edd518 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:00:45.352630 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ef7dc3-5669-43da-8923-0ac221edd518 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:00:45.352909 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ef7dc3-5669-43da-8923-0ac221edd518 None admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:00:45.353673 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ef7dc3-5669-43da-8923-0ac221edd518 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 05:00:45.353984 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ef7dc3-5669-43da-8923-0ac221edd518 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:00:45.355356 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ef7dc3-5669-43da-8923-0ac221edd518 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:00:45.363120 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 605/1189] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Jun 18 05:00:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 05:00:45.369034 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040dbcd2-fd9d-4995-98d4-b8d84b785650 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:00:45.474049 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040dbcd2-fd9d-4995-98d4-b8d84b785650 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:00:45.507630 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f1f0d8-49c0-4803-9d3c-3de6b4bb24aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:00:45.508476 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f1f0d8-49c0-4803-9d3c-3de6b4bb24aa None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:00:45.508755 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f1f0d8-49c0-4803-9d3c-3de6b4bb24aa None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:00:45.509025 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f1f0d8-49c0-4803-9d3c-3de6b4bb24aa None admin] PATH_INFO: `/v3/users/4fa34bd49f1e46a699e051a9d9ff6a72` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:00:45.510227 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f1f0d8-49c0-4803-9d3c-3de6b4bb24aa None admin] RBAC: Authorizing `identity:get_user(user_id=4fa34bd49f1e46a699e051a9d9ff6a72)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:00:45.511636 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f1f0d8-49c0-4803-9d3c-3de6b4bb24aa None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:00:45.513235 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 585/1190] 10.222.0.23 () {56 vars in 1177 bytes} [Sat Jun 18 05:00:45 2022] GET /identity/v3/users/4fa34bd49f1e46a699e051a9d9ff6a72 => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 05:00:45.519078 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ef7dc3-5669-43da-8923-0ac221edd518 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:00:45.547126 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:00:45.618776 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ef7dc3-5669-43da-8923-0ac221edd518 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:00:45.652418 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6fa8c1-040a-4d30-96f2-c6a910b89e48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:00:45.653251 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6fa8c1-040a-4d30-96f2-c6a910b89e48 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:00:45.653558 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6fa8c1-040a-4d30-96f2-c6a910b89e48 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:00:45.653829 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6fa8c1-040a-4d30-96f2-c6a910b89e48 None admin] PATH_INFO: `/v3/projects/f58c08e7bed64e45b1eb401b297e97d7` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:00:45.654985 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6fa8c1-040a-4d30-96f2-c6a910b89e48 None admin] RBAC: Authorizing `identity:get_project(project_id=f58c08e7bed64e45b1eb401b297e97d7)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:00:45.656399 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6fa8c1-040a-4d30-96f2-c6a910b89e48 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:00:45.658107 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 606/1191] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Jun 18 05:00:45 2022] GET /identity/v3/projects/f58c08e7bed64e45b1eb401b297e97d7 => generated 361 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 05:00:45.664478 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f1f0d8-49c0-4803-9d3c-3de6b4bb24aa None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:00:45.724632 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:00:45.763501 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f1f0d8-49c0-4803-9d3c-3de6b4bb24aa None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:00:45.797763 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a80289d-34a2-4f6b-b59a-a9565643ac14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:00:45.798629 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a80289d-34a2-4f6b-b59a-a9565643ac14 None admin] REQUEST_METHOD: `PUT` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:00:45.798911 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a80289d-34a2-4f6b-b59a-a9565643ac14 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:00:45.799188 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a80289d-34a2-4f6b-b59a-a9565643ac14 None admin] PATH_INFO: `/v3/projects/f58c08e7bed64e45b1eb401b297e97d7/users/4fa34bd49f1e46a699e051a9d9ff6a72/roles/21c65f2c51bd4dd283d13507d2714467` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:00:45.801154 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a80289d-34a2-4f6b-b59a-a9565643ac14 None admin] RBAC: Authorizing `identity:create_grant(project_id=f58c08e7bed64e45b1eb401b297e97d7, user_id=4fa34bd49f1e46a699e051a9d9ff6a72, role_id=21c65f2c51bd4dd283d13507d2714467)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:00:45.802697 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a80289d-34a2-4f6b-b59a-a9565643ac14 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:00:45.895512 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 586/1192] 10.222.0.23 () {58 vars in 1439 bytes} [Sat Jun 18 05:00:45 2022] PUT /identity/v3/projects/f58c08e7bed64e45b1eb401b297e97d7/users/4fa34bd49f1e46a699e051a9d9ff6a72/roles/21c65f2c51bd4dd283d13507d2714467 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:00:47.943584 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7a6fa3-2e0b-4fab-9ae0-c52a76aba378 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:00:47.944167 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7a6fa3-2e0b-4fab-9ae0-c52a76aba378 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:00:47.944167 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7a6fa3-2e0b-4fab-9ae0-c52a76aba378 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:00:47.945634 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 607/1193] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 05:00:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 05:00:47.950890 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488ef7a3-572e-4db9-98d6-6fb6fb07c9ae None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:00:47.951302 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488ef7a3-572e-4db9-98d6-6fb6fb07c9ae None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:00:47.951450 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488ef7a3-572e-4db9-98d6-6fb6fb07c9ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:00:47.968343 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-488ef7a3-572e-4db9-98d6-6fb6fb07c9ae None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:00:47.969558 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-488ef7a3-572e-4db9-98d6-6fb6fb07c9ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:00:48.034289 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 587/1194] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 05:00:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 18 05:00:48.058311 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7a6fa3-2e0b-4fab-9ae0-c52a76aba378 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:00:48.151238 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:00:48.158503 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7a6fa3-2e0b-4fab-9ae0-c52a76aba378 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:00:48.191939 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efd0e05-dc97-499c-9b6e-c7288fb7fe85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:00:48.192704 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efd0e05-dc97-499c-9b6e-c7288fb7fe85 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:00:48.192985 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efd0e05-dc97-499c-9b6e-c7288fb7fe85 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:00:48.193260 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efd0e05-dc97-499c-9b6e-c7288fb7fe85 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:00:48.203430 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efd0e05-dc97-499c-9b6e-c7288fb7fe85 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 18 05:00:48.212239 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-0efd0e05-dc97-499c-9b6e-c7288fb7fe85 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 18 05:00:48.212239 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 05:00:48.212239 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 05:00:48.212239 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 05:00:48.212239 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 05:00:48.212239 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 05:00:48.212239 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 05:00:48.212239 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 05:00:48.212239 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 05:00:48.212239 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 05:00:48.212239 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 05:00:48.212239 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 05:00:48.212239 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 18 05:00:48.212239 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 18 05:00:48.212239 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 18 05:00:48.212239 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 18 05:00:48.213322 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 18 05:00:48.213322 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 18 05:00:48.213322 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 18 05:00:48.213322 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 18 05:00:48.213322 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 05:00:48.213322 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 05:00:48.213322 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 05:00:48.213322 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 05:00:48.213322 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 05:00:48.213322 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 05:00:48.213322 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 05:00:48.213322 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jun 18 05:00:48.213322 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 05:00:48.213322 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jun 18 05:00:48.213322 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 05:00:48.213322 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 05:00:48.214386 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 05:00:48.214386 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jun 18 05:00:48.214386 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 05:00:48.214386 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jun 18 05:00:48.214386 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 18 05:00:48.214386 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 18 05:00:48.214386 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 18 05:00:48.214386 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 18 05:00:48.214386 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 18 05:00:48.214386 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 18 05:00:48.214386 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 18 05:00:48.214386 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 05:00:48.214386 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 608/1195] 10.222.0.23 () {56 vars in 1096 bytes} [Sat Jun 18 05:00:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 05:00:48.219387 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488ef7a3-572e-4db9-98d6-6fb6fb07c9ae None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:00:48.271644 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:00:48.317240 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488ef7a3-572e-4db9-98d6-6fb6fb07c9ae None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:00:48.351532 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2c39f7-c5a6-4726-8070-df1f68e04827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:00:48.352179 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2c39f7-c5a6-4726-8070-df1f68e04827 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:00:48.352464 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2c39f7-c5a6-4726-8070-df1f68e04827 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:00:48.352740 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2c39f7-c5a6-4726-8070-df1f68e04827 None admin] PATH_INFO: `/v3/roles` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:00:48.353503 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2c39f7-c5a6-4726-8070-df1f68e04827 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 05:00:48.353814 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2c39f7-c5a6-4726-8070-df1f68e04827 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:00:48.355171 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2c39f7-c5a6-4726-8070-df1f68e04827 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:00:48.362975 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 588/1196] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Jun 18 05:00:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 05:00:48.369561 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efd0e05-dc97-499c-9b6e-c7288fb7fe85 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:00:48.465680 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efd0e05-dc97-499c-9b6e-c7288fb7fe85 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:00:48.498625 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b412186-cbf0-4d43-995f-adbf0ef99966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:00:48.499473 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b412186-cbf0-4d43-995f-adbf0ef99966 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:00:48.499760 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b412186-cbf0-4d43-995f-adbf0ef99966 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:00:48.500035 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b412186-cbf0-4d43-995f-adbf0ef99966 None admin] PATH_INFO: `/v3/users/4fa34bd49f1e46a699e051a9d9ff6a72` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:00:48.501200 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b412186-cbf0-4d43-995f-adbf0ef99966 None admin] RBAC: Authorizing `identity:get_user(user_id=4fa34bd49f1e46a699e051a9d9ff6a72)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:00:48.502676 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b412186-cbf0-4d43-995f-adbf0ef99966 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:00:48.504289 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 609/1197] 10.222.0.23 () {56 vars in 1177 bytes} [Sat Jun 18 05:00:48 2022] GET /identity/v3/users/4fa34bd49f1e46a699e051a9d9ff6a72 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 05:00:48.510302 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2c39f7-c5a6-4726-8070-df1f68e04827 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:00:48.608046 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2c39f7-c5a6-4726-8070-df1f68e04827 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:00:48.622156 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:00:48.641560 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5975500c-2536-40fc-8eaa-9e6d748692ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:00:48.642194 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5975500c-2536-40fc-8eaa-9e6d748692ca None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:00:48.642550 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5975500c-2536-40fc-8eaa-9e6d748692ca None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:00:48.642829 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5975500c-2536-40fc-8eaa-9e6d748692ca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:00:48.709605 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5975500c-2536-40fc-8eaa-9e6d748692ca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:00:48.711209 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5975500c-2536-40fc-8eaa-9e6d748692ca None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:00:48.808371 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 589/1198] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Jun 18 05:00:48 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 18 05:00:48.814292 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b412186-cbf0-4d43-995f-adbf0ef99966 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:00:48.839176 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:00:48.913309 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b412186-cbf0-4d43-995f-adbf0ef99966 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:00:48.946150 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acb2007-d3c7-4cfd-a500-63cb1dc6113f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:00:48.946983 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acb2007-d3c7-4cfd-a500-63cb1dc6113f None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:00:48.947260 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acb2007-d3c7-4cfd-a500-63cb1dc6113f None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:00:48.947524 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acb2007-d3c7-4cfd-a500-63cb1dc6113f None admin] PATH_INFO: `/v3/projects/f58c08e7bed64e45b1eb401b297e97d7` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:00:48.948681 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acb2007-d3c7-4cfd-a500-63cb1dc6113f None admin] RBAC: Authorizing `identity:get_project(project_id=f58c08e7bed64e45b1eb401b297e97d7)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:00:48.950125 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acb2007-d3c7-4cfd-a500-63cb1dc6113f None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:00:48.951827 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 610/1199] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Jun 18 05:00:48 2022] GET /identity/v3/projects/f58c08e7bed64e45b1eb401b297e97d7 => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 05:00:48.957840 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5975500c-2536-40fc-8eaa-9e6d748692ca None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:00:48.972832 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:00:49.055454 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5975500c-2536-40fc-8eaa-9e6d748692ca None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:00:49.089026 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09089f4-47dd-4a29-9c89-98023ebf6de4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:00:49.089622 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09089f4-47dd-4a29-9c89-98023ebf6de4 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:00:49.089904 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09089f4-47dd-4a29-9c89-98023ebf6de4 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:00:49.090178 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09089f4-47dd-4a29-9c89-98023ebf6de4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:00:49.090996 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09089f4-47dd-4a29-9c89-98023ebf6de4 None admin] RBAC: Adding query filter params (role.id=21c65f2c51bd4dd283d13507d2714467, scope.project.id=f58c08e7bed64e45b1eb401b297e97d7, user.id=4fa34bd49f1e46a699e051a9d9ff6a72) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 05:00:49.091316 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09089f4-47dd-4a29-9c89-98023ebf6de4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:00:49.092730 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09089f4-47dd-4a29-9c89-98023ebf6de4 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:00:49.098439 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 590/1200] 10.222.0.23 () {56 vars in 1374 bytes} [Sat Jun 18 05:00:48 2022] GET /identity/v3/role_assignments?user.id=4fa34bd49f1e46a699e051a9d9ff6a72&scope.project.id=f58c08e7bed64e45b1eb401b297e97d7&role.id=21c65f2c51bd4dd283d13507d2714467 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 18 05:00:58.941539 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=55 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:00:59.095794 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=61 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:02:01.185329 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f3cc6d-7b96-46d4-be17-a53d2854a2c7 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:02:01.185927 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f3cc6d-7b96-46d4-be17-a53d2854a2c7 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:02:01.185927 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f3cc6d-7b96-46d4-be17-a53d2854a2c7 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:02:01.187293 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 611/1201] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 05:02:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 05:02:01.192547 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cc9c42-c9c9-47aa-b987-0f7810162dd7 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:02:01.198761 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cc9c42-c9c9-47aa-b987-0f7810162dd7 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:02:01.198761 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cc9c42-c9c9-47aa-b987-0f7810162dd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:02:01.210502 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-50cc9c42-c9c9-47aa-b987-0f7810162dd7 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:02:01.211746 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-50cc9c42-c9c9-47aa-b987-0f7810162dd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:02:01.278430 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 591/1202] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 05:02:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 18 05:02:01.303104 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f3cc6d-7b96-46d4-be17-a53d2854a2c7 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:02:01.409140 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f3cc6d-7b96-46d4-be17-a53d2854a2c7 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:02:01.441467 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f637de7e-556e-47b1-bb8f-f4b42f136abd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:02:01.442191 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f637de7e-556e-47b1-bb8f-f4b42f136abd None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:02:01.442520 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f637de7e-556e-47b1-bb8f-f4b42f136abd None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:02:01.442794 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f637de7e-556e-47b1-bb8f-f4b42f136abd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:02:01.453518 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f637de7e-556e-47b1-bb8f-f4b42f136abd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:02:01.454956 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f637de7e-556e-47b1-bb8f-f4b42f136abd None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:02:01.471927 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-f637de7e-556e-47b1-bb8f-f4b42f136abd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 05:02:01.471927 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 05:02:01.471927 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 05:02:01.471927 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 05:02:01.471927 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 05:02:01.471927 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 05:02:01.471927 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 05:02:01.471927 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 05:02:01.471927 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 05:02:01.471927 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 05:02:01.471927 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 05:02:01.471927 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 05:02:01.471927 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 18 05:02:01.471927 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 18 05:02:01.471927 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 18 05:02:01.478040 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 18 05:02:01.478040 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 05:02:01.478040 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 05:02:01.478040 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 05:02:01.478040 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 05:02:01.478040 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 05:02:01.478040 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 05:02:01.478040 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 05:02:01.478040 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jun 18 05:02:01.478040 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 05:02:01.478040 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jun 18 05:02:01.478040 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 05:02:01.478040 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 05:02:01.478040 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 05:02:01.478040 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jun 18 05:02:01.478040 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 05:02:01.478040 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jun 18 05:02:01.479171 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 18 05:02:01.479171 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 18 05:02:01.479171 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 05:02:01.479171 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 05:02:01.479171 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 612/1203] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Jun 18 05:02:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 05:02:01.479534 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cc9c42-c9c9-47aa-b987-0f7810162dd7 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:02:01.577770 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cc9c42-c9c9-47aa-b987-0f7810162dd7 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:02:01.606225 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:02:01.612013 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da88ea5a-bce1-4205-a908-2b35fe42b028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:02:01.612667 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da88ea5a-bce1-4205-a908-2b35fe42b028 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:02:01.612953 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da88ea5a-bce1-4205-a908-2b35fe42b028 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:02:01.613229 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da88ea5a-bce1-4205-a908-2b35fe42b028 None admin] PATH_INFO: `/v3/domains` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:02:01.614007 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da88ea5a-bce1-4205-a908-2b35fe42b028 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 05:02:01.614349 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da88ea5a-bce1-4205-a908-2b35fe42b028 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:02:01.615710 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da88ea5a-bce1-4205-a908-2b35fe42b028 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:02:01.627028 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 592/1204] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Jun 18 05:02:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 05:02:01.632656 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f637de7e-556e-47b1-bb8f-f4b42f136abd None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:02:01.686201 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:02:01.733356 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f637de7e-556e-47b1-bb8f-f4b42f136abd None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:02:01.767805 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70c3ad0-79cd-48ea-b97e-2199c527e6a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:02:01.768461 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70c3ad0-79cd-48ea-b97e-2199c527e6a6 None admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:02:01.768743 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70c3ad0-79cd-48ea-b97e-2199c527e6a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:02:01.769015 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70c3ad0-79cd-48ea-b97e-2199c527e6a6 None admin] PATH_INFO: `/v3/users` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:02:01.769738 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70c3ad0-79cd-48ea-b97e-2199c527e6a6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:02:01.771170 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70c3ad0-79cd-48ea-b97e-2199c527e6a6 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:02:01.894776 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 613/1205] 10.222.0.23 () {60 vars in 1132 bytes} [Sat Jun 18 05:02:01 2022] POST /identity/v3/users => generated 251 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:02:03.987465 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2081af-1db5-49c0-bdc8-aca7334cbae3 None None] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:02:03.988017 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2081af-1db5-49c0-bdc8-aca7334cbae3 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:02:03.988017 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2081af-1db5-49c0-bdc8-aca7334cbae3 None None] PATH_INFO: `/` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:02:03.989411 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 593/1206] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 05:02:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 18 05:02:03.994749 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1d7485-0ec1-4969-bf40-d3ff1cabfa22 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:02:03.995437 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1d7485-0ec1-4969-bf40-d3ff1cabfa22 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:02:03.995549 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1d7485-0ec1-4969-bf40-d3ff1cabfa22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:02:04.012951 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-6a1d7485-0ec1-4969-bf40-d3ff1cabfa22 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:02:04.014183 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-6a1d7485-0ec1-4969-bf40-d3ff1cabfa22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:02:04.080250 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 614/1207] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 05:02:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 18 05:02:04.104439 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2081af-1db5-49c0-bdc8-aca7334cbae3 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:02:04.208272 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2081af-1db5-49c0-bdc8-aca7334cbae3 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:02:04.240753 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e920ae6-edda-41b2-8896-56938eb50402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:02:04.241551 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e920ae6-edda-41b2-8896-56938eb50402 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:02:04.241832 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e920ae6-edda-41b2-8896-56938eb50402 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:02:04.242102 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e920ae6-edda-41b2-8896-56938eb50402 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:02:04.254141 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e920ae6-edda-41b2-8896-56938eb50402 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 18 05:02:04.264517 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application [None req-8e920ae6-edda-41b2-8896-56938eb50402 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 18 05:02:04.264517 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 05:02:04.264517 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 05:02:04.264517 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 05:02:04.264517 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 05:02:04.264517 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 05:02:04.264517 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 05:02:04.264517 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 05:02:04.264517 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 05:02:04.264517 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 05:02:04.264517 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 05:02:04.264517 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 05:02:04.264517 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 18 05:02:04.264517 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 18 05:02:04.264517 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 18 05:02:04.264517 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 18 05:02:04.271421 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 18 05:02:04.271421 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 18 05:02:04.271421 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 18 05:02:04.271421 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 18 05:02:04.271421 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 05:02:04.271421 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 05:02:04.271421 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 05:02:04.271421 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 05:02:04.271421 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 05:02:04.271421 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 05:02:04.271421 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 05:02:04.271421 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application with Lock( Jun 18 05:02:04.271421 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 05:02:04.271421 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._enter() Jun 18 05:02:04.271421 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 05:02:04.271421 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 05:02:04.272293 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 05:02:04.272293 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.creator() Jun 18 05:02:04.272293 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 05:02:04.272293 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application created_value = creator( Jun 18 05:02:04.272293 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 18 05:02:04.272293 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 18 05:02:04.272293 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 18 05:02:04.272293 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 18 05:02:04.272293 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 18 05:02:04.272293 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 18 05:02:04.272293 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 18 05:02:04.272293 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 05:02:04.272293 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 594/1208] 10.222.0.23 () {56 vars in 1102 bytes} [Sat Jun 18 05:02:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 05:02:04.273024 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1d7485-0ec1-4969-bf40-d3ff1cabfa22 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:02:04.313000 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:02:04.371450 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1d7485-0ec1-4969-bf40-d3ff1cabfa22 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:02:04.405424 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23185d07-cdab-4365-89ea-0258c5ed185e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:02:04.406090 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23185d07-cdab-4365-89ea-0258c5ed185e None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:02:04.406373 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23185d07-cdab-4365-89ea-0258c5ed185e None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:02:04.406648 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23185d07-cdab-4365-89ea-0258c5ed185e None admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:02:04.407383 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23185d07-cdab-4365-89ea-0258c5ed185e None admin] RBAC: Adding query filter params (name=service) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 05:02:04.407690 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23185d07-cdab-4365-89ea-0258c5ed185e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:02:04.409164 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23185d07-cdab-4365-89ea-0258c5ed185e None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:02:04.417408 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 615/1209] 10.222.0.23 () {56 vars in 1103 bytes} [Sat Jun 18 05:02:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 05:02:04.423220 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e920ae6-edda-41b2-8896-56938eb50402 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:02:04.472945 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:02:04.524413 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e920ae6-edda-41b2-8896-56938eb50402 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:02:04.558824 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384af007-e1be-4929-83cb-cc264d7478aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:02:04.559587 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384af007-e1be-4929-83cb-cc264d7478aa None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:02:04.559866 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384af007-e1be-4929-83cb-cc264d7478aa None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:02:04.560139 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384af007-e1be-4929-83cb-cc264d7478aa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:02:04.570010 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384af007-e1be-4929-83cb-cc264d7478aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:02:04.572151 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384af007-e1be-4929-83cb-cc264d7478aa None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:02:04.589689 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application [None req-384af007-e1be-4929-83cb-cc264d7478aa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 05:02:04.589689 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 05:02:04.589689 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 05:02:04.589689 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 05:02:04.589689 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 05:02:04.589689 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 05:02:04.589689 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 05:02:04.589689 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 05:02:04.589689 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 05:02:04.589689 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 05:02:04.589689 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 05:02:04.589689 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 05:02:04.589689 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 18 05:02:04.589689 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 18 05:02:04.589689 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 18 05:02:04.591893 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 18 05:02:04.591893 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 05:02:04.591893 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 05:02:04.591893 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 05:02:04.591893 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 05:02:04.591893 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 05:02:04.591893 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 05:02:04.591893 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 05:02:04.591893 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application with Lock( Jun 18 05:02:04.591893 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 05:02:04.591893 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._enter() Jun 18 05:02:04.591893 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 05:02:04.591893 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 05:02:04.591893 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 05:02:04.591893 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.creator() Jun 18 05:02:04.591893 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 05:02:04.591893 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application created_value = creator( Jun 18 05:02:04.593115 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 18 05:02:04.593115 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 18 05:02:04.593115 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 05:02:04.593115 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 05:02:04.593115 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 595/1210] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Jun 18 05:02:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 05:02:04.596851 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23185d07-cdab-4365-89ea-0258c5ed185e None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:02:04.692628 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:02:04.696093 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23185d07-cdab-4365-89ea-0258c5ed185e None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:02:04.728561 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf598229-fc7c-41d9-8864-2525b5f85327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:02:04.729210 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf598229-fc7c-41d9-8864-2525b5f85327 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:02:04.729521 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf598229-fc7c-41d9-8864-2525b5f85327 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:02:04.729798 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf598229-fc7c-41d9-8864-2525b5f85327 None admin] PATH_INFO: `/v3/domains` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:02:04.730507 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf598229-fc7c-41d9-8864-2525b5f85327 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 05:02:04.730816 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf598229-fc7c-41d9-8864-2525b5f85327 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:02:04.732185 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf598229-fc7c-41d9-8864-2525b5f85327 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:02:04.743657 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 616/1211] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Jun 18 05:02:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 05:02:04.749636 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384af007-e1be-4929-83cb-cc264d7478aa None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:02:04.849331 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384af007-e1be-4929-83cb-cc264d7478aa None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:02:04.850837 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:02:04.883628 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581f6675-f3bc-4cbf-97df-05652b4c8a70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:02:04.884486 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581f6675-f3bc-4cbf-97df-05652b4c8a70 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:02:04.884775 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581f6675-f3bc-4cbf-97df-05652b4c8a70 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:02:04.885052 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581f6675-f3bc-4cbf-97df-05652b4c8a70 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:02:04.891911 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581f6675-f3bc-4cbf-97df-05652b4c8a70 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:02:04.893344 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581f6675-f3bc-4cbf-97df-05652b4c8a70 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:02:04.907672 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application [None req-581f6675-f3bc-4cbf-97df-05652b4c8a70 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 18 05:02:04.907672 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 05:02:04.907672 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 05:02:04.907672 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 05:02:04.907672 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 05:02:04.907672 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 05:02:04.907672 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 05:02:04.907672 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 05:02:04.907672 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 05:02:04.907672 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 05:02:04.907672 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 05:02:04.907672 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 05:02:04.907672 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 18 05:02:04.907672 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 18 05:02:04.907672 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 18 05:02:04.907672 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 18 05:02:04.908790 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 05:02:04.908790 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 05:02:04.908790 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 18 05:02:04.908790 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 18 05:02:04.908790 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 18 05:02:04.908790 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 18 05:02:04.908790 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 05:02:04.908790 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 05:02:04.908790 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 05:02:04.908790 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 05:02:04.908790 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 05:02:04.908790 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application with Lock( Jun 18 05:02:04.908790 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 05:02:04.908790 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._enter() Jun 18 05:02:04.908790 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 05:02:04.908790 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 05:02:04.908790 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 05:02:04.908790 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.creator() Jun 18 05:02:04.910257 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 05:02:04.910257 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application created_value = creator( Jun 18 05:02:04.910257 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 18 05:02:04.910257 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 18 05:02:04.910257 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 18 05:02:04.910257 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 18 05:02:04.910257 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 18 05:02:04.910257 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 18 05:02:04.910257 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 18 05:02:04.910257 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 05:02:04.910257 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 596/1212] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Jun 18 05:02:04 2022] GET /identity/v3/users/placement => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 05:02:04.914722 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf598229-fc7c-41d9-8864-2525b5f85327 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:02:05.014022 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf598229-fc7c-41d9-8864-2525b5f85327 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:02:05.047177 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81429304-bc56-4491-9bdf-2e586c35af49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:02:05.047850 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81429304-bc56-4491-9bdf-2e586c35af49 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:02:05.048167 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81429304-bc56-4491-9bdf-2e586c35af49 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:02:05.048449 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81429304-bc56-4491-9bdf-2e586c35af49 None admin] PATH_INFO: `/v3/users` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:02:05.049264 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81429304-bc56-4491-9bdf-2e586c35af49 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 05:02:05.049598 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81429304-bc56-4491-9bdf-2e586c35af49 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:02:05.051011 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81429304-bc56-4491-9bdf-2e586c35af49 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:02:05.068568 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 617/1213] 10.222.0.23 () {56 vars in 1143 bytes} [Sat Jun 18 05:02:04 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 05:02:05.075053 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581f6675-f3bc-4cbf-97df-05652b4c8a70 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:02:05.175695 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581f6675-f3bc-4cbf-97df-05652b4c8a70 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:02:05.213556 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf68e7c-b47c-49f4-bb05-4b671a08b7ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:02:05.214188 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf68e7c-b47c-49f4-bb05-4b671a08b7ba None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:02:05.214469 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf68e7c-b47c-49f4-bb05-4b671a08b7ba None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:02:05.214745 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf68e7c-b47c-49f4-bb05-4b671a08b7ba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:02:05.246404 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:02:05.284235 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf68e7c-b47c-49f4-bb05-4b671a08b7ba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:02:05.285711 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf68e7c-b47c-49f4-bb05-4b671a08b7ba None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:02:05.386747 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 597/1214] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Jun 18 05:02:05 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 18 05:02:05.392499 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81429304-bc56-4491-9bdf-2e586c35af49 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:02:05.411173 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:02:05.491611 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81429304-bc56-4491-9bdf-2e586c35af49 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:02:05.526300 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4789f4-691b-436e-a250-d5c5161277e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:02:05.527094 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4789f4-691b-436e-a250-d5c5161277e6 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:02:05.527382 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4789f4-691b-436e-a250-d5c5161277e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:02:05.527656 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4789f4-691b-436e-a250-d5c5161277e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:02:05.538142 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4789f4-691b-436e-a250-d5c5161277e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:02:05.540284 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4789f4-691b-436e-a250-d5c5161277e6 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:02:05.558585 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-6c4789f4-691b-436e-a250-d5c5161277e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 05:02:05.558585 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 05:02:05.558585 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 05:02:05.558585 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 05:02:05.558585 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 05:02:05.558585 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 05:02:05.558585 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 05:02:05.558585 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 05:02:05.558585 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 05:02:05.558585 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 05:02:05.558585 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 05:02:05.558585 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 05:02:05.558585 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 18 05:02:05.558585 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 18 05:02:05.558585 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 18 05:02:05.559675 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 18 05:02:05.559675 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 05:02:05.559675 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 05:02:05.559675 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 05:02:05.559675 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 05:02:05.559675 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 05:02:05.559675 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 05:02:05.559675 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 05:02:05.559675 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jun 18 05:02:05.559675 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 05:02:05.559675 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jun 18 05:02:05.559675 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 05:02:05.559675 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 05:02:05.559675 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 05:02:05.559675 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jun 18 05:02:05.559675 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 05:02:05.559675 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jun 18 05:02:05.570325 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 18 05:02:05.570325 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 18 05:02:05.570325 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 05:02:05.570325 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 05:02:05.570325 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 618/1215] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Jun 18 05:02:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 05:02:05.570744 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf68e7c-b47c-49f4-bb05-4b671a08b7ba None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:02:05.627613 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:02:05.666265 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf68e7c-b47c-49f4-bb05-4b671a08b7ba None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:02:05.699719 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dae621-0ab0-4900-acb4-0672144e83f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:02:05.700428 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dae621-0ab0-4900-acb4-0672144e83f6 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:02:05.700713 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dae621-0ab0-4900-acb4-0672144e83f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:02:05.700990 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dae621-0ab0-4900-acb4-0672144e83f6 None admin] PATH_INFO: `/v3/domains` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:02:05.701734 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61dae621-0ab0-4900-acb4-0672144e83f6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 05:02:05.702047 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61dae621-0ab0-4900-acb4-0672144e83f6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:02:05.703395 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61dae621-0ab0-4900-acb4-0672144e83f6 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:02:05.715146 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 598/1216] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Jun 18 05:02:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 05:02:05.720770 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4789f4-691b-436e-a250-d5c5161277e6 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:02:05.790614 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:02:05.819715 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4789f4-691b-436e-a250-d5c5161277e6 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:02:05.855276 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9c5f12-b985-4579-a874-e119438b64ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:02:05.856077 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9c5f12-b985-4579-a874-e119438b64ca None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:02:05.856358 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9c5f12-b985-4579-a874-e119438b64ca None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:02:05.856628 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9c5f12-b985-4579-a874-e119438b64ca None admin] PATH_INFO: `/v3/projects/service` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:02:05.863100 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9c5f12-b985-4579-a874-e119438b64ca None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:02:05.864535 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9c5f12-b985-4579-a874-e119438b64ca None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:02:05.878299 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-bf9c5f12-b985-4579-a874-e119438b64ca None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 18 05:02:05.878299 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 05:02:05.878299 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 05:02:05.878299 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 05:02:05.878299 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 05:02:05.878299 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 05:02:05.878299 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 05:02:05.878299 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 05:02:05.878299 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 05:02:05.878299 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 05:02:05.878299 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 05:02:05.878299 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 05:02:05.878299 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 18 05:02:05.878299 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 18 05:02:05.878299 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 18 05:02:05.879355 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 18 05:02:05.879355 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 05:02:05.879355 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 05:02:05.879355 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 05:02:05.879355 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 05:02:05.879355 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 05:02:05.879355 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 05:02:05.879355 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 05:02:05.879355 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jun 18 05:02:05.879355 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 05:02:05.879355 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jun 18 05:02:05.879355 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 05:02:05.879355 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 05:02:05.879355 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 05:02:05.879355 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jun 18 05:02:05.879355 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 05:02:05.879355 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jun 18 05:02:05.880623 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 18 05:02:05.880623 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 18 05:02:05.880623 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 18 05:02:05.880623 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 18 05:02:05.880623 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 18 05:02:05.880623 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 18 05:02:05.880623 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 18 05:02:05.880623 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 05:02:05.880623 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 619/1217] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Jun 18 05:02:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 05:02:05.885915 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dae621-0ab0-4900-acb4-0672144e83f6 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:02:05.986691 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dae621-0ab0-4900-acb4-0672144e83f6 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:02:06.009054 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:02:06.020075 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc32d0ff-7555-4ab2-9514-585b45d63cab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:02:06.020720 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc32d0ff-7555-4ab2-9514-585b45d63cab None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:02:06.021002 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc32d0ff-7555-4ab2-9514-585b45d63cab None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:02:06.021274 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc32d0ff-7555-4ab2-9514-585b45d63cab None admin] PATH_INFO: `/v3/projects` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:02:06.022095 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc32d0ff-7555-4ab2-9514-585b45d63cab None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 05:02:06.022407 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc32d0ff-7555-4ab2-9514-585b45d63cab None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:02:06.024513 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc32d0ff-7555-4ab2-9514-585b45d63cab None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:02:06.036017 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 599/1218] 10.222.0.23 () {56 vars in 1148 bytes} [Sat Jun 18 05:02:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 05:02:06.042366 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9c5f12-b985-4579-a874-e119438b64ca None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:02:06.143072 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9c5f12-b985-4579-a874-e119438b64ca None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:02:06.171272 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:02:06.176871 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844d46cc-3e53-466a-9f02-9c3fe5e67ad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:02:06.177537 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844d46cc-3e53-466a-9f02-9c3fe5e67ad0 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:02:06.177813 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844d46cc-3e53-466a-9f02-9c3fe5e67ad0 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:02:06.178082 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844d46cc-3e53-466a-9f02-9c3fe5e67ad0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:02:06.178895 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844d46cc-3e53-466a-9f02-9c3fe5e67ad0 None admin] RBAC: Adding query filter params (role.id=5ec7b68b793d4a54abd30460961346b9, scope.project.id=c95f8f3084c5472cb8364b7c57207394, user.id=5323ffd91f864680b0c0084eb3f44d21) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 05:02:06.179204 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844d46cc-3e53-466a-9f02-9c3fe5e67ad0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:02:06.180647 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844d46cc-3e53-466a-9f02-9c3fe5e67ad0 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:02:06.185925 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 620/1219] 10.222.0.23 () {56 vars in 1374 bytes} [Sat Jun 18 05:02:06 2022] GET /identity/v3/role_assignments?user.id=5323ffd91f864680b0c0084eb3f44d21&scope.project.id=c95f8f3084c5472cb8364b7c57207394&role.id=5ec7b68b793d4a54abd30460961346b9 => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 05:02:08.203491 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cf4ad9-ef19-4c96-9b0f-c513bc0efa7a None None] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:02:08.204006 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cf4ad9-ef19-4c96-9b0f-c513bc0efa7a None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:02:08.204094 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cf4ad9-ef19-4c96-9b0f-c513bc0efa7a None None] PATH_INFO: `/` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:02:08.205429 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 600/1220] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 05:02:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 05:02:08.211027 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb37254-d1fe-4d24-83f2-8faf1c6f5c67 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:02:08.211490 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb37254-d1fe-4d24-83f2-8faf1c6f5c67 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:02:08.211596 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb37254-d1fe-4d24-83f2-8faf1c6f5c67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:02:08.229629 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-cbb37254-d1fe-4d24-83f2-8faf1c6f5c67 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:02:08.230837 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-cbb37254-d1fe-4d24-83f2-8faf1c6f5c67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:02:08.300461 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 621/1221] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 05:02:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 18 05:02:08.324325 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cf4ad9-ef19-4c96-9b0f-c513bc0efa7a None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:02:08.432617 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cf4ad9-ef19-4c96-9b0f-c513bc0efa7a None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:02:08.466524 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7143a01c-dca3-49d7-aab2-723d3006644c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:02:08.467237 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7143a01c-dca3-49d7-aab2-723d3006644c None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:02:08.467517 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7143a01c-dca3-49d7-aab2-723d3006644c None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:02:08.467787 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7143a01c-dca3-49d7-aab2-723d3006644c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:02:08.478262 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7143a01c-dca3-49d7-aab2-723d3006644c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 18 05:02:08.485873 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application [None req-7143a01c-dca3-49d7-aab2-723d3006644c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 18 05:02:08.485873 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 05:02:08.485873 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 05:02:08.485873 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 05:02:08.485873 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 05:02:08.485873 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 05:02:08.485873 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 05:02:08.485873 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 05:02:08.485873 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 05:02:08.485873 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 05:02:08.485873 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 05:02:08.485873 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 05:02:08.485873 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 18 05:02:08.485873 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 18 05:02:08.485873 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 18 05:02:08.485873 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 18 05:02:08.486858 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 18 05:02:08.486858 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 18 05:02:08.486858 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 18 05:02:08.486858 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 18 05:02:08.486858 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 05:02:08.486858 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 05:02:08.486858 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 05:02:08.486858 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 05:02:08.486858 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 05:02:08.486858 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 05:02:08.486858 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 05:02:08.486858 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application with Lock( Jun 18 05:02:08.486858 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 05:02:08.486858 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._enter() Jun 18 05:02:08.486858 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 05:02:08.486858 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 05:02:08.487764 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 05:02:08.487764 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.creator() Jun 18 05:02:08.487764 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 05:02:08.487764 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application created_value = creator( Jun 18 05:02:08.487764 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 18 05:02:08.487764 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 18 05:02:08.487764 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 18 05:02:08.487764 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 18 05:02:08.487764 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 18 05:02:08.487764 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 18 05:02:08.487764 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 18 05:02:08.487764 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 05:02:08.487764 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 601/1222] 10.222.0.23 () {56 vars in 1102 bytes} [Sat Jun 18 05:02:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 05:02:08.493937 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb37254-d1fe-4d24-83f2-8faf1c6f5c67 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:02:08.592495 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb37254-d1fe-4d24-83f2-8faf1c6f5c67 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:02:08.615843 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:02:08.626460 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ff5188-08a0-4fdb-b09e-45b35deaf989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:02:08.627106 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ff5188-08a0-4fdb-b09e-45b35deaf989 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:02:08.627399 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ff5188-08a0-4fdb-b09e-45b35deaf989 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:02:08.627683 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ff5188-08a0-4fdb-b09e-45b35deaf989 None admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:02:08.628417 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ff5188-08a0-4fdb-b09e-45b35deaf989 None admin] RBAC: Adding query filter params (name=service) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 05:02:08.628726 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ff5188-08a0-4fdb-b09e-45b35deaf989 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:02:08.630114 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ff5188-08a0-4fdb-b09e-45b35deaf989 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:02:08.638027 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 622/1223] 10.222.0.23 () {56 vars in 1103 bytes} [Sat Jun 18 05:02:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 05:02:08.644035 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7143a01c-dca3-49d7-aab2-723d3006644c None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:02:08.688849 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:02:08.746627 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7143a01c-dca3-49d7-aab2-723d3006644c None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:02:08.780567 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580f65d4-ecfe-4d2c-95ab-1d3ed3b68563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:02:08.781344 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580f65d4-ecfe-4d2c-95ab-1d3ed3b68563 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:02:08.781667 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580f65d4-ecfe-4d2c-95ab-1d3ed3b68563 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:02:08.781937 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580f65d4-ecfe-4d2c-95ab-1d3ed3b68563 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:02:08.792233 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580f65d4-ecfe-4d2c-95ab-1d3ed3b68563 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:02:08.797756 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580f65d4-ecfe-4d2c-95ab-1d3ed3b68563 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:02:08.810662 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application [None req-580f65d4-ecfe-4d2c-95ab-1d3ed3b68563 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 05:02:08.810662 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 05:02:08.810662 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 05:02:08.810662 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 05:02:08.810662 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 05:02:08.810662 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 05:02:08.810662 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 05:02:08.810662 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 05:02:08.810662 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 05:02:08.810662 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 05:02:08.810662 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 05:02:08.810662 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 05:02:08.810662 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 18 05:02:08.810662 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 18 05:02:08.810662 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 18 05:02:08.811592 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 18 05:02:08.811592 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 05:02:08.811592 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 05:02:08.811592 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 05:02:08.811592 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 05:02:08.811592 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 05:02:08.811592 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 05:02:08.811592 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 05:02:08.811592 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application with Lock( Jun 18 05:02:08.811592 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 05:02:08.811592 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._enter() Jun 18 05:02:08.811592 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 05:02:08.811592 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 05:02:08.811592 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 05:02:08.811592 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.creator() Jun 18 05:02:08.811592 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 05:02:08.811592 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application created_value = creator( Jun 18 05:02:08.812918 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 18 05:02:08.812918 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 18 05:02:08.812918 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 05:02:08.812918 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 05:02:08.812918 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 602/1224] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Jun 18 05:02:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 05:02:08.817611 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ff5188-08a0-4fdb-b09e-45b35deaf989 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:02:08.916414 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ff5188-08a0-4fdb-b09e-45b35deaf989 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:02:08.949264 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b55284-c6b4-48ff-9539-8344c584b549 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:02:08.949935 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b55284-c6b4-48ff-9539-8344c584b549 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:02:08.950224 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b55284-c6b4-48ff-9539-8344c584b549 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:02:08.950497 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b55284-c6b4-48ff-9539-8344c584b549 None admin] PATH_INFO: `/v3/domains` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:02:08.951204 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b55284-c6b4-48ff-9539-8344c584b549 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 05:02:08.951549 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b55284-c6b4-48ff-9539-8344c584b549 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:02:08.952898 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b55284-c6b4-48ff-9539-8344c584b549 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:02:08.964436 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 623/1225] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Jun 18 05:02:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 05:02:08.970128 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580f65d4-ecfe-4d2c-95ab-1d3ed3b68563 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:02:09.067478 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:02:09.074102 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580f65d4-ecfe-4d2c-95ab-1d3ed3b68563 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:02:09.108193 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88529780-7192-4eb0-9f65-da98950ce32d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:02:09.108981 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88529780-7192-4eb0-9f65-da98950ce32d None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:02:09.109280 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88529780-7192-4eb0-9f65-da98950ce32d None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:02:09.109594 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88529780-7192-4eb0-9f65-da98950ce32d None admin] PATH_INFO: `/v3/users/placement` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:02:09.116468 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88529780-7192-4eb0-9f65-da98950ce32d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:02:09.118650 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88529780-7192-4eb0-9f65-da98950ce32d None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:02:09.133426 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application [None req-88529780-7192-4eb0-9f65-da98950ce32d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 18 05:02:09.133426 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 05:02:09.133426 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 05:02:09.133426 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 05:02:09.133426 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 05:02:09.133426 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 05:02:09.133426 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 05:02:09.133426 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 05:02:09.133426 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 05:02:09.133426 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 05:02:09.133426 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 05:02:09.133426 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 05:02:09.133426 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 18 05:02:09.133426 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 18 05:02:09.133426 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 18 05:02:09.133426 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 18 05:02:09.136336 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 05:02:09.136336 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 05:02:09.136336 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 18 05:02:09.136336 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 18 05:02:09.136336 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 18 05:02:09.136336 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 18 05:02:09.136336 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 05:02:09.136336 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 05:02:09.136336 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 05:02:09.136336 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 05:02:09.136336 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 05:02:09.136336 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application with Lock( Jun 18 05:02:09.136336 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 05:02:09.136336 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._enter() Jun 18 05:02:09.136336 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 05:02:09.136336 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 05:02:09.136336 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 05:02:09.136336 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.creator() Jun 18 05:02:09.137522 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 05:02:09.137522 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application created_value = creator( Jun 18 05:02:09.137522 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 18 05:02:09.137522 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 18 05:02:09.137522 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 18 05:02:09.137522 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 18 05:02:09.137522 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 18 05:02:09.137522 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 18 05:02:09.137522 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 18 05:02:09.137522 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 05:02:09.137522 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 603/1226] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Jun 18 05:02:08 2022] GET /identity/v3/users/placement => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 05:02:09.140545 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b55284-c6b4-48ff-9539-8344c584b549 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:02:09.187526 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:02:09.238946 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b55284-c6b4-48ff-9539-8344c584b549 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:02:09.274431 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45ce4cf-0c55-458b-9ef7-ebdb13c150df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:02:09.275084 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45ce4cf-0c55-458b-9ef7-ebdb13c150df None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:02:09.275365 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45ce4cf-0c55-458b-9ef7-ebdb13c150df None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:02:09.275668 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45ce4cf-0c55-458b-9ef7-ebdb13c150df None admin] PATH_INFO: `/v3/users` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:02:09.276486 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45ce4cf-0c55-458b-9ef7-ebdb13c150df None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 05:02:09.276796 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45ce4cf-0c55-458b-9ef7-ebdb13c150df None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:02:09.278281 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45ce4cf-0c55-458b-9ef7-ebdb13c150df None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:02:09.296500 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 624/1227] 10.222.0.23 () {56 vars in 1143 bytes} [Sat Jun 18 05:02:09 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 05:02:09.302165 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88529780-7192-4eb0-9f65-da98950ce32d None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:02:09.402938 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88529780-7192-4eb0-9f65-da98950ce32d None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:02:09.436665 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae50454-8ac2-42a2-9cd1-c20dea5ca6aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:02:09.437476 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae50454-8ac2-42a2-9cd1-c20dea5ca6aa None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:02:09.437757 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae50454-8ac2-42a2-9cd1-c20dea5ca6aa None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:02:09.438026 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae50454-8ac2-42a2-9cd1-c20dea5ca6aa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:02:09.448603 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae50454-8ac2-42a2-9cd1-c20dea5ca6aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:02:09.450053 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae50454-8ac2-42a2-9cd1-c20dea5ca6aa None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:02:09.466986 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application [None req-2ae50454-8ac2-42a2-9cd1-c20dea5ca6aa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 05:02:09.466986 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 05:02:09.466986 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 05:02:09.466986 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 05:02:09.466986 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 05:02:09.466986 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 05:02:09.466986 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 05:02:09.466986 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 05:02:09.466986 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 05:02:09.466986 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 05:02:09.466986 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 05:02:09.466986 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 05:02:09.466986 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 18 05:02:09.466986 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 18 05:02:09.466986 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 18 05:02:09.471511 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 18 05:02:09.471511 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 05:02:09.471511 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 05:02:09.471511 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 05:02:09.471511 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 05:02:09.471511 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 05:02:09.471511 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 05:02:09.471511 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 05:02:09.471511 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application with Lock( Jun 18 05:02:09.471511 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 05:02:09.471511 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._enter() Jun 18 05:02:09.471511 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 05:02:09.471511 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 05:02:09.471511 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 05:02:09.471511 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.creator() Jun 18 05:02:09.471511 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 05:02:09.471511 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application created_value = creator( Jun 18 05:02:09.472604 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 18 05:02:09.472604 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 18 05:02:09.472604 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 05:02:09.472604 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 05:02:09.472604 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 604/1228] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Jun 18 05:02:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 05:02:09.473932 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45ce4cf-0c55-458b-9ef7-ebdb13c150df None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:02:09.572495 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45ce4cf-0c55-458b-9ef7-ebdb13c150df None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:02:09.574852 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:02:09.606222 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad45913-97f6-4174-88db-c34ee040bba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:02:09.606933 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad45913-97f6-4174-88db-c34ee040bba5 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:02:09.607209 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad45913-97f6-4174-88db-c34ee040bba5 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:02:09.607482 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad45913-97f6-4174-88db-c34ee040bba5 None admin] PATH_INFO: `/v3/domains` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:02:09.608428 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad45913-97f6-4174-88db-c34ee040bba5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 05:02:09.608732 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad45913-97f6-4174-88db-c34ee040bba5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:02:09.610234 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad45913-97f6-4174-88db-c34ee040bba5 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:02:09.622647 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 625/1229] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Jun 18 05:02:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 05:02:09.628404 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae50454-8ac2-42a2-9cd1-c20dea5ca6aa None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:02:09.637776 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:02:09.728244 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae50454-8ac2-42a2-9cd1-c20dea5ca6aa None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:02:09.763841 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45adf523-b8ea-49c7-82c8-654670888954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:02:09.764620 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45adf523-b8ea-49c7-82c8-654670888954 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:02:09.764902 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45adf523-b8ea-49c7-82c8-654670888954 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:02:09.765176 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45adf523-b8ea-49c7-82c8-654670888954 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:02:09.771812 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45adf523-b8ea-49c7-82c8-654670888954 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:02:09.773234 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45adf523-b8ea-49c7-82c8-654670888954 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:02:09.786570 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application [None req-45adf523-b8ea-49c7-82c8-654670888954 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 18 05:02:09.786570 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 05:02:09.786570 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 05:02:09.786570 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 05:02:09.786570 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 05:02:09.786570 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 05:02:09.786570 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 05:02:09.786570 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 05:02:09.786570 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 05:02:09.786570 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 05:02:09.786570 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 05:02:09.786570 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 05:02:09.786570 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 18 05:02:09.786570 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 18 05:02:09.786570 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 18 05:02:09.787621 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 18 05:02:09.787621 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 05:02:09.787621 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 05:02:09.787621 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 05:02:09.787621 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 05:02:09.787621 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 05:02:09.787621 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 05:02:09.787621 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 05:02:09.787621 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application with Lock( Jun 18 05:02:09.787621 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 05:02:09.787621 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._enter() Jun 18 05:02:09.787621 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 05:02:09.787621 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 05:02:09.787621 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 05:02:09.787621 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.creator() Jun 18 05:02:09.787621 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 05:02:09.787621 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application created_value = creator( Jun 18 05:02:09.788584 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 18 05:02:09.788584 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 18 05:02:09.788584 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 18 05:02:09.788584 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 18 05:02:09.788584 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 18 05:02:09.788584 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 18 05:02:09.788584 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 18 05:02:09.788584 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 05:02:09.788584 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 605/1230] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Jun 18 05:02:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 05:02:09.794421 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad45913-97f6-4174-88db-c34ee040bba5 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:02:09.895153 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad45913-97f6-4174-88db-c34ee040bba5 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:02:09.928447 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e035c8a-9384-4278-8112-64f9bc2cd4b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:02:09.929087 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e035c8a-9384-4278-8112-64f9bc2cd4b6 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:02:09.929365 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e035c8a-9384-4278-8112-64f9bc2cd4b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:02:09.929666 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e035c8a-9384-4278-8112-64f9bc2cd4b6 None admin] PATH_INFO: `/v3/projects` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:02:09.931280 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e035c8a-9384-4278-8112-64f9bc2cd4b6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 05:02:09.931590 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e035c8a-9384-4278-8112-64f9bc2cd4b6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:02:09.933022 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e035c8a-9384-4278-8112-64f9bc2cd4b6 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:02:09.944661 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 626/1231] 10.222.0.23 () {56 vars in 1148 bytes} [Sat Jun 18 05:02:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 05:02:09.951069 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45adf523-b8ea-49c7-82c8-654670888954 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:02:10.018482 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:02:10.053191 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45adf523-b8ea-49c7-82c8-654670888954 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:02:10.090436 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90febe0f-640b-4a2f-bcda-19d6c86437e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:02:10.091328 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90febe0f-640b-4a2f-bcda-19d6c86437e3 None admin] REQUEST_METHOD: `PUT` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:02:10.091628 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90febe0f-640b-4a2f-bcda-19d6c86437e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:02:10.091923 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90febe0f-640b-4a2f-bcda-19d6c86437e3 None admin] PATH_INFO: `/v3/projects/c95f8f3084c5472cb8364b7c57207394/users/5323ffd91f864680b0c0084eb3f44d21/roles/5ec7b68b793d4a54abd30460961346b9` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:02:10.106512 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90febe0f-640b-4a2f-bcda-19d6c86437e3 None admin] RBAC: Authorizing `identity:create_grant(project_id=c95f8f3084c5472cb8364b7c57207394, user_id=5323ffd91f864680b0c0084eb3f44d21, role_id=5ec7b68b793d4a54abd30460961346b9)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:02:10.108038 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90febe0f-640b-4a2f-bcda-19d6c86437e3 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:02:10.163518 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 606/1232] 10.222.0.23 () {58 vars in 1439 bytes} [Sat Jun 18 05:02:09 2022] PUT /identity/v3/projects/c95f8f3084c5472cb8364b7c57207394/users/5323ffd91f864680b0c0084eb3f44d21/roles/5ec7b68b793d4a54abd30460961346b9 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:02:12.198412 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743d3f19-ab84-43e9-a21b-64580c5cdf9c None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:02:12.198970 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743d3f19-ab84-43e9-a21b-64580c5cdf9c None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:02:12.198970 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743d3f19-ab84-43e9-a21b-64580c5cdf9c None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:02:12.200370 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 627/1233] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 05:02:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 05:02:12.206813 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d63f0d-2336-4de8-b295-6c2b9428ec71 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:02:12.207253 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d63f0d-2336-4de8-b295-6c2b9428ec71 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:02:12.207369 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d63f0d-2336-4de8-b295-6c2b9428ec71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:02:12.224536 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-c5d63f0d-2336-4de8-b295-6c2b9428ec71 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:02:12.225756 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-c5d63f0d-2336-4de8-b295-6c2b9428ec71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:02:12.291619 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 607/1234] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 05:02:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 18 05:02:12.320020 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743d3f19-ab84-43e9-a21b-64580c5cdf9c None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:02:12.347327 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:02:12.424011 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743d3f19-ab84-43e9-a21b-64580c5cdf9c None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:02:12.457782 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6524688-be44-4122-bb16-cda91d02b829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:02:12.458506 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6524688-be44-4122-bb16-cda91d02b829 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:02:12.458792 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6524688-be44-4122-bb16-cda91d02b829 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:02:12.459070 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6524688-be44-4122-bb16-cda91d02b829 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:02:12.469927 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6524688-be44-4122-bb16-cda91d02b829 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 18 05:02:12.478555 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-b6524688-be44-4122-bb16-cda91d02b829 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 18 05:02:12.478555 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 05:02:12.478555 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 05:02:12.478555 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 05:02:12.478555 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 05:02:12.478555 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 05:02:12.478555 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 05:02:12.478555 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 05:02:12.478555 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 05:02:12.478555 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 05:02:12.478555 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 05:02:12.478555 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 05:02:12.478555 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 18 05:02:12.478555 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 18 05:02:12.478555 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 18 05:02:12.478555 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 18 05:02:12.479727 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 18 05:02:12.479727 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 18 05:02:12.479727 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 18 05:02:12.479727 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 18 05:02:12.479727 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 05:02:12.479727 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 05:02:12.479727 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 05:02:12.479727 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 05:02:12.479727 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 05:02:12.479727 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 05:02:12.479727 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 05:02:12.479727 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jun 18 05:02:12.479727 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 05:02:12.479727 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jun 18 05:02:12.479727 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 05:02:12.479727 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 05:02:12.480991 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 05:02:12.480991 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jun 18 05:02:12.480991 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 05:02:12.480991 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jun 18 05:02:12.480991 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 18 05:02:12.480991 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 18 05:02:12.480991 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 18 05:02:12.480991 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 18 05:02:12.480991 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 18 05:02:12.480991 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 18 05:02:12.480991 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 18 05:02:12.480991 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 05:02:12.480991 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 628/1235] 10.222.0.23 () {56 vars in 1102 bytes} [Sat Jun 18 05:02:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 05:02:12.485797 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d63f0d-2336-4de8-b295-6c2b9428ec71 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:02:12.533361 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:02:12.588720 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d63f0d-2336-4de8-b295-6c2b9428ec71 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:02:12.621938 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136cde51-27ba-4c69-b597-e4b5f770a4d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:02:12.622590 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136cde51-27ba-4c69-b597-e4b5f770a4d2 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:02:12.622871 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136cde51-27ba-4c69-b597-e4b5f770a4d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:02:12.623141 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136cde51-27ba-4c69-b597-e4b5f770a4d2 None admin] PATH_INFO: `/v3/roles` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:02:12.623870 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136cde51-27ba-4c69-b597-e4b5f770a4d2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 05:02:12.624175 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136cde51-27ba-4c69-b597-e4b5f770a4d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:02:12.625559 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136cde51-27ba-4c69-b597-e4b5f770a4d2 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:02:12.634589 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 608/1236] 10.222.0.23 () {56 vars in 1103 bytes} [Sat Jun 18 05:02:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 05:02:12.640330 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6524688-be44-4122-bb16-cda91d02b829 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:02:12.728839 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:02:12.745847 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6524688-be44-4122-bb16-cda91d02b829 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:02:12.779572 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3137ed-0634-4646-b356-155123a5994b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:02:12.780344 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3137ed-0634-4646-b356-155123a5994b None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:02:12.780625 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3137ed-0634-4646-b356-155123a5994b None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:02:12.780895 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3137ed-0634-4646-b356-155123a5994b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:02:12.791904 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3137ed-0634-4646-b356-155123a5994b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:02:12.793341 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3137ed-0634-4646-b356-155123a5994b None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:02:12.810806 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-fc3137ed-0634-4646-b356-155123a5994b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 05:02:12.810806 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 05:02:12.810806 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 05:02:12.810806 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 05:02:12.810806 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 05:02:12.810806 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 05:02:12.810806 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 05:02:12.810806 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 05:02:12.810806 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 05:02:12.810806 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 05:02:12.810806 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 05:02:12.810806 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 05:02:12.810806 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 18 05:02:12.810806 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 18 05:02:12.810806 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 18 05:02:12.811866 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 18 05:02:12.811866 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 05:02:12.811866 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 05:02:12.811866 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 05:02:12.811866 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 05:02:12.811866 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 05:02:12.811866 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 05:02:12.811866 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 05:02:12.811866 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jun 18 05:02:12.811866 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 05:02:12.811866 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jun 18 05:02:12.811866 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 05:02:12.811866 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 05:02:12.811866 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 05:02:12.811866 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jun 18 05:02:12.811866 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 05:02:12.811866 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jun 18 05:02:12.812829 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 18 05:02:12.812829 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 18 05:02:12.812829 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 05:02:12.812829 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 05:02:12.812829 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 629/1237] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Jun 18 05:02:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 05:02:12.817948 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136cde51-27ba-4c69-b597-e4b5f770a4d2 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:02:12.917314 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136cde51-27ba-4c69-b597-e4b5f770a4d2 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:02:12.926613 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:02:12.951177 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5361221-bd40-48b2-bb61-3a819f745a6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:02:12.951821 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5361221-bd40-48b2-bb61-3a819f745a6b None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:02:12.952104 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5361221-bd40-48b2-bb61-3a819f745a6b None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:02:12.952377 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5361221-bd40-48b2-bb61-3a819f745a6b None admin] PATH_INFO: `/v3/domains` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:02:12.953080 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5361221-bd40-48b2-bb61-3a819f745a6b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 05:02:12.953411 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5361221-bd40-48b2-bb61-3a819f745a6b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:02:12.954798 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5361221-bd40-48b2-bb61-3a819f745a6b None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:02:12.965850 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 609/1238] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Jun 18 05:02:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 05:02:12.971689 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3137ed-0634-4646-b356-155123a5994b None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:02:13.074380 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3137ed-0634-4646-b356-155123a5994b None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:02:13.107293 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcc65db-51cd-4c39-9490-cf594efe53c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:02:13.108212 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcc65db-51cd-4c39-9490-cf594efe53c9 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:02:13.108495 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcc65db-51cd-4c39-9490-cf594efe53c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:02:13.108770 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcc65db-51cd-4c39-9490-cf594efe53c9 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:02:13.111474 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:02:13.116589 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcc65db-51cd-4c39-9490-cf594efe53c9 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:02:13.118788 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcc65db-51cd-4c39-9490-cf594efe53c9 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:02:13.133762 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-6fcc65db-51cd-4c39-9490-cf594efe53c9 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 18 05:02:13.133762 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 05:02:13.133762 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 05:02:13.133762 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 05:02:13.133762 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 05:02:13.133762 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 05:02:13.133762 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 05:02:13.133762 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 05:02:13.133762 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 05:02:13.133762 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 05:02:13.133762 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 05:02:13.133762 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 05:02:13.133762 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 18 05:02:13.133762 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 18 05:02:13.133762 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 18 05:02:13.133762 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 18 05:02:13.134793 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 05:02:13.134793 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 05:02:13.134793 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 18 05:02:13.134793 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 18 05:02:13.134793 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 18 05:02:13.134793 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 18 05:02:13.134793 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 05:02:13.134793 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 05:02:13.134793 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 05:02:13.134793 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 05:02:13.134793 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 05:02:13.134793 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jun 18 05:02:13.134793 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 05:02:13.134793 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jun 18 05:02:13.134793 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 05:02:13.134793 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 05:02:13.134793 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 05:02:13.134793 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jun 18 05:02:13.135751 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 05:02:13.135751 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jun 18 05:02:13.135751 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 18 05:02:13.135751 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 18 05:02:13.135751 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 18 05:02:13.135751 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 18 05:02:13.135751 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 18 05:02:13.135751 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 18 05:02:13.135751 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 18 05:02:13.135751 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 05:02:13.135751 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 630/1239] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Jun 18 05:02:12 2022] GET /identity/v3/users/placement => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 05:02:13.140958 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5361221-bd40-48b2-bb61-3a819f745a6b None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:02:13.240147 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5361221-bd40-48b2-bb61-3a819f745a6b None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:02:13.274185 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611551ba-7367-4198-8040-0b5a133ac608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:02:13.274838 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611551ba-7367-4198-8040-0b5a133ac608 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:02:13.275115 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611551ba-7367-4198-8040-0b5a133ac608 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:02:13.275388 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611551ba-7367-4198-8040-0b5a133ac608 None admin] PATH_INFO: `/v3/users` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:02:13.276333 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611551ba-7367-4198-8040-0b5a133ac608 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 05:02:13.276588 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611551ba-7367-4198-8040-0b5a133ac608 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:02:13.278027 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611551ba-7367-4198-8040-0b5a133ac608 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:02:13.295489 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 610/1240] 10.222.0.23 () {56 vars in 1143 bytes} [Sat Jun 18 05:02:13 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 05:02:13.301122 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcc65db-51cd-4c39-9490-cf594efe53c9 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:02:13.401887 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcc65db-51cd-4c39-9490-cf594efe53c9 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:02:13.435061 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3013d671-97df-494e-bc06-98c41bff19da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:02:13.435692 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3013d671-97df-494e-bc06-98c41bff19da None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:02:13.435972 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3013d671-97df-494e-bc06-98c41bff19da None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:02:13.436287 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3013d671-97df-494e-bc06-98c41bff19da None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:02:13.496236 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:02:13.502585 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3013d671-97df-494e-bc06-98c41bff19da None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:02:13.504078 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3013d671-97df-494e-bc06-98c41bff19da None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:02:13.604403 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 631/1241] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Jun 18 05:02:13 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 18 05:02:13.605595 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611551ba-7367-4198-8040-0b5a133ac608 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:02:13.637284 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:02:13.707534 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611551ba-7367-4198-8040-0b5a133ac608 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:02:13.740182 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9880de-9b8a-4717-9bfe-024c5019fb01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:02:13.740949 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9880de-9b8a-4717-9bfe-024c5019fb01 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:02:13.741228 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9880de-9b8a-4717-9bfe-024c5019fb01 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:02:13.741527 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9880de-9b8a-4717-9bfe-024c5019fb01 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:02:13.752342 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9880de-9b8a-4717-9bfe-024c5019fb01 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:02:13.754462 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9880de-9b8a-4717-9bfe-024c5019fb01 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:02:13.774041 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application [None req-bf9880de-9b8a-4717-9bfe-024c5019fb01 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 05:02:13.774041 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 05:02:13.774041 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 05:02:13.774041 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 05:02:13.774041 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 05:02:13.774041 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 05:02:13.774041 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 05:02:13.774041 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 05:02:13.774041 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 05:02:13.774041 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 05:02:13.774041 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 05:02:13.774041 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 05:02:13.774041 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 18 05:02:13.774041 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 18 05:02:13.774041 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 18 05:02:13.775125 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 18 05:02:13.775125 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 05:02:13.775125 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 05:02:13.775125 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 05:02:13.775125 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 05:02:13.775125 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 05:02:13.775125 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 05:02:13.775125 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 05:02:13.775125 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application with Lock( Jun 18 05:02:13.775125 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 05:02:13.775125 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._enter() Jun 18 05:02:13.775125 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 05:02:13.775125 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 05:02:13.775125 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 05:02:13.775125 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.creator() Jun 18 05:02:13.775125 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 05:02:13.775125 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application created_value = creator( Jun 18 05:02:13.776233 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 18 05:02:13.776233 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 18 05:02:13.776233 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 05:02:13.776233 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 05:02:13.776233 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 611/1242] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Jun 18 05:02:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 05:02:13.781028 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3013d671-97df-494e-bc06-98c41bff19da None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:02:13.873993 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:02:13.879623 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3013d671-97df-494e-bc06-98c41bff19da None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:02:13.912637 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7f58a2-f1f4-4b0a-ac49-83c92f945263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:02:13.913281 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7f58a2-f1f4-4b0a-ac49-83c92f945263 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:02:13.913602 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7f58a2-f1f4-4b0a-ac49-83c92f945263 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:02:13.913879 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7f58a2-f1f4-4b0a-ac49-83c92f945263 None admin] PATH_INFO: `/v3/domains` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:02:13.915187 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7f58a2-f1f4-4b0a-ac49-83c92f945263 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 05:02:13.915497 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7f58a2-f1f4-4b0a-ac49-83c92f945263 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:02:13.916901 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7f58a2-f1f4-4b0a-ac49-83c92f945263 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:02:13.927741 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 632/1243] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Jun 18 05:02:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 05:02:13.933424 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9880de-9b8a-4717-9bfe-024c5019fb01 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:02:14.013974 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:02:14.035007 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9880de-9b8a-4717-9bfe-024c5019fb01 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:02:14.068194 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0694f135-b7a7-4707-a674-6f049b3bc8e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:02:14.068993 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0694f135-b7a7-4707-a674-6f049b3bc8e4 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:02:14.069334 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0694f135-b7a7-4707-a674-6f049b3bc8e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:02:14.069644 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0694f135-b7a7-4707-a674-6f049b3bc8e4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:02:14.077155 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0694f135-b7a7-4707-a674-6f049b3bc8e4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:02:14.078625 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0694f135-b7a7-4707-a674-6f049b3bc8e4 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:02:14.092625 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application [None req-0694f135-b7a7-4707-a674-6f049b3bc8e4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 18 05:02:14.092625 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 05:02:14.092625 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 05:02:14.092625 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 05:02:14.092625 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 05:02:14.092625 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 05:02:14.092625 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 05:02:14.092625 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 05:02:14.092625 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 05:02:14.092625 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 05:02:14.092625 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 05:02:14.092625 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 05:02:14.092625 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 18 05:02:14.092625 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 18 05:02:14.092625 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 18 05:02:14.093734 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 18 05:02:14.093734 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 05:02:14.093734 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 05:02:14.093734 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 05:02:14.093734 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 05:02:14.093734 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 05:02:14.093734 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 05:02:14.093734 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 05:02:14.093734 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application with Lock( Jun 18 05:02:14.093734 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 05:02:14.093734 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._enter() Jun 18 05:02:14.093734 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 05:02:14.093734 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 05:02:14.093734 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 05:02:14.093734 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.creator() Jun 18 05:02:14.093734 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 05:02:14.093734 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application created_value = creator( Jun 18 05:02:14.094774 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 18 05:02:14.094774 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 18 05:02:14.094774 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 18 05:02:14.094774 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 18 05:02:14.094774 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 18 05:02:14.094774 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 18 05:02:14.094774 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 18 05:02:14.094774 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 05:02:14.094774 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 612/1244] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Jun 18 05:02:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 05:02:14.099641 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7f58a2-f1f4-4b0a-ac49-83c92f945263 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:02:14.201408 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7f58a2-f1f4-4b0a-ac49-83c92f945263 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:02:14.235414 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18df2c8a-2c2c-48a2-aefd-b1e219982ab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:02:14.236050 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18df2c8a-2c2c-48a2-aefd-b1e219982ab3 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:02:14.236329 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18df2c8a-2c2c-48a2-aefd-b1e219982ab3 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:02:14.236598 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18df2c8a-2c2c-48a2-aefd-b1e219982ab3 None admin] PATH_INFO: `/v3/projects` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:02:14.237363 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18df2c8a-2c2c-48a2-aefd-b1e219982ab3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 05:02:14.237693 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18df2c8a-2c2c-48a2-aefd-b1e219982ab3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:02:14.239119 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18df2c8a-2c2c-48a2-aefd-b1e219982ab3 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:02:14.252239 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 633/1245] 10.222.0.23 () {56 vars in 1148 bytes} [Sat Jun 18 05:02:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 05:02:14.258296 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0694f135-b7a7-4707-a674-6f049b3bc8e4 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:02:14.357917 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0694f135-b7a7-4707-a674-6f049b3bc8e4 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:02:14.391840 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32419c25-0c1c-492a-a8c4-e8dc38011f27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:02:14.392482 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32419c25-0c1c-492a-a8c4-e8dc38011f27 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:02:14.392763 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32419c25-0c1c-492a-a8c4-e8dc38011f27 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:02:14.393034 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32419c25-0c1c-492a-a8c4-e8dc38011f27 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:02:14.393895 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32419c25-0c1c-492a-a8c4-e8dc38011f27 None admin] RBAC: Adding query filter params (role.id=5ec7b68b793d4a54abd30460961346b9, scope.project.id=c95f8f3084c5472cb8364b7c57207394, user.id=5323ffd91f864680b0c0084eb3f44d21) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 05:02:14.394206 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32419c25-0c1c-492a-a8c4-e8dc38011f27 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:02:14.395616 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32419c25-0c1c-492a-a8c4-e8dc38011f27 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:02:14.401459 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 613/1246] 10.222.0.23 () {56 vars in 1374 bytes} [Sat Jun 18 05:02:14 2022] GET /identity/v3/role_assignments?user.id=5323ffd91f864680b0c0084eb3f44d21&scope.project.id=c95f8f3084c5472cb8364b7c57207394&role.id=5ec7b68b793d4a54abd30460961346b9 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 05:02:16.473811 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e7141f-c270-4ccc-a82c-56d4045f26b2 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:02:16.474348 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e7141f-c270-4ccc-a82c-56d4045f26b2 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:02:16.474348 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e7141f-c270-4ccc-a82c-56d4045f26b2 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:02:16.475789 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 634/1247] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 05:02:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 05:02:16.481400 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef9a540-db0d-4d02-953e-ea7773212b5b None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:02:16.481802 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef9a540-db0d-4d02-953e-ea7773212b5b None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:02:16.481972 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef9a540-db0d-4d02-953e-ea7773212b5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:02:16.486986 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:02:16.499644 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-5ef9a540-db0d-4d02-953e-ea7773212b5b None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:02:16.500870 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-5ef9a540-db0d-4d02-953e-ea7773212b5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:02:16.567171 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 614/1248] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 05:02:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 18 05:02:16.590542 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e7141f-c270-4ccc-a82c-56d4045f26b2 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:02:16.608813 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:02:16.693179 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e7141f-c270-4ccc-a82c-56d4045f26b2 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:02:16.727230 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e721a42-50ba-4e08-b05f-6f3ebe25fcb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:02:16.728063 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e721a42-50ba-4e08-b05f-6f3ebe25fcb9 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:02:16.728355 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e721a42-50ba-4e08-b05f-6f3ebe25fcb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:02:16.728623 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e721a42-50ba-4e08-b05f-6f3ebe25fcb9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:02:16.738578 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e721a42-50ba-4e08-b05f-6f3ebe25fcb9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 18 05:02:16.747186 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-8e721a42-50ba-4e08-b05f-6f3ebe25fcb9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 18 05:02:16.747186 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 05:02:16.747186 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 05:02:16.747186 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 05:02:16.747186 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 05:02:16.747186 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 05:02:16.747186 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 05:02:16.747186 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 05:02:16.747186 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 05:02:16.747186 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 05:02:16.747186 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 05:02:16.747186 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 05:02:16.747186 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 18 05:02:16.747186 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 18 05:02:16.747186 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 18 05:02:16.747186 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 18 05:02:16.748294 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 18 05:02:16.748294 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 18 05:02:16.748294 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 18 05:02:16.748294 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 18 05:02:16.748294 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 05:02:16.748294 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 05:02:16.748294 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 05:02:16.748294 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 05:02:16.748294 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 05:02:16.748294 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 05:02:16.748294 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 05:02:16.748294 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jun 18 05:02:16.748294 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 05:02:16.748294 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jun 18 05:02:16.748294 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 05:02:16.748294 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 05:02:16.749320 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 05:02:16.749320 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jun 18 05:02:16.749320 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 05:02:16.749320 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jun 18 05:02:16.749320 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 18 05:02:16.749320 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 18 05:02:16.749320 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 18 05:02:16.749320 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 18 05:02:16.749320 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 18 05:02:16.749320 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 18 05:02:16.749320 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 18 05:02:16.749320 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 05:02:16.749320 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 635/1249] 10.222.0.23 () {56 vars in 1096 bytes} [Sat Jun 18 05:02:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 05:02:16.754337 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef9a540-db0d-4d02-953e-ea7773212b5b None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:02:16.851218 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef9a540-db0d-4d02-953e-ea7773212b5b None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:02:16.880771 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:02:16.885828 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141c0fd2-86e4-4be4-89e6-096c0bb69baf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:02:16.886485 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141c0fd2-86e4-4be4-89e6-096c0bb69baf None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:02:16.886768 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141c0fd2-86e4-4be4-89e6-096c0bb69baf None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:02:16.887043 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141c0fd2-86e4-4be4-89e6-096c0bb69baf None admin] PATH_INFO: `/v3/roles` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:02:16.887781 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141c0fd2-86e4-4be4-89e6-096c0bb69baf None admin] RBAC: Adding query filter params (name=admin) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 05:02:16.888085 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141c0fd2-86e4-4be4-89e6-096c0bb69baf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:02:16.889461 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141c0fd2-86e4-4be4-89e6-096c0bb69baf None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:02:16.897206 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 615/1250] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Jun 18 05:02:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 05:02:16.903871 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e721a42-50ba-4e08-b05f-6f3ebe25fcb9 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:02:16.986927 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:02:17.009386 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e721a42-50ba-4e08-b05f-6f3ebe25fcb9 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:02:17.042930 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419f5d74-27e4-4654-8b3f-2d631bd10357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:02:17.043709 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419f5d74-27e4-4654-8b3f-2d631bd10357 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:02:17.043986 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419f5d74-27e4-4654-8b3f-2d631bd10357 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:02:17.044253 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419f5d74-27e4-4654-8b3f-2d631bd10357 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:02:17.054553 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419f5d74-27e4-4654-8b3f-2d631bd10357 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:02:17.056292 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419f5d74-27e4-4654-8b3f-2d631bd10357 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:02:17.073239 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-419f5d74-27e4-4654-8b3f-2d631bd10357 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 05:02:17.073239 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 05:02:17.073239 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 05:02:17.073239 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 05:02:17.073239 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 05:02:17.073239 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 05:02:17.073239 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 05:02:17.073239 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 05:02:17.073239 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 05:02:17.073239 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 05:02:17.073239 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 05:02:17.073239 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 05:02:17.073239 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 18 05:02:17.073239 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 18 05:02:17.073239 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 18 05:02:17.074363 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 18 05:02:17.074363 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 05:02:17.074363 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 05:02:17.074363 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 05:02:17.074363 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 05:02:17.074363 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 05:02:17.074363 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 05:02:17.074363 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 05:02:17.074363 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jun 18 05:02:17.074363 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 05:02:17.074363 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jun 18 05:02:17.074363 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 05:02:17.074363 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 05:02:17.074363 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 05:02:17.074363 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jun 18 05:02:17.074363 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 05:02:17.074363 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jun 18 05:02:17.075451 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 18 05:02:17.075451 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 18 05:02:17.075451 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 05:02:17.075451 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 05:02:17.075451 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 636/1251] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Jun 18 05:02:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 05:02:17.080334 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141c0fd2-86e4-4be4-89e6-096c0bb69baf None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:02:17.182204 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141c0fd2-86e4-4be4-89e6-096c0bb69baf None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:02:17.214951 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb04b789-e616-4d17-9b6d-57364d2d2ddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:02:17.215603 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb04b789-e616-4d17-9b6d-57364d2d2ddc None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:02:17.215889 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb04b789-e616-4d17-9b6d-57364d2d2ddc None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:02:17.216167 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb04b789-e616-4d17-9b6d-57364d2d2ddc None admin] PATH_INFO: `/v3/domains` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:02:17.216881 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb04b789-e616-4d17-9b6d-57364d2d2ddc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 05:02:17.217193 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb04b789-e616-4d17-9b6d-57364d2d2ddc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:02:17.218599 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb04b789-e616-4d17-9b6d-57364d2d2ddc None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:02:17.230803 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 616/1252] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Jun 18 05:02:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 05:02:17.236480 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419f5d74-27e4-4654-8b3f-2d631bd10357 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:02:17.342194 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419f5d74-27e4-4654-8b3f-2d631bd10357 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:02:17.369465 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:02:17.377508 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fd891d-403d-483f-b2c5-633da6756b6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:02:17.378307 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fd891d-403d-483f-b2c5-633da6756b6d None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:02:17.378586 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fd891d-403d-483f-b2c5-633da6756b6d None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:02:17.378855 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fd891d-403d-483f-b2c5-633da6756b6d None admin] PATH_INFO: `/v3/users/placement` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:02:17.386973 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fd891d-403d-483f-b2c5-633da6756b6d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:02:17.390874 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fd891d-403d-483f-b2c5-633da6756b6d None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:02:17.402709 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-27fd891d-403d-483f-b2c5-633da6756b6d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 18 05:02:17.402709 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 05:02:17.402709 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 05:02:17.402709 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 05:02:17.402709 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 05:02:17.402709 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 05:02:17.402709 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 05:02:17.402709 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 05:02:17.402709 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 05:02:17.402709 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 05:02:17.402709 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 05:02:17.402709 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 05:02:17.402709 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 18 05:02:17.402709 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 18 05:02:17.402709 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 18 05:02:17.402709 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 18 05:02:17.403817 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 05:02:17.403817 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 05:02:17.403817 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 18 05:02:17.403817 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 18 05:02:17.403817 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 18 05:02:17.403817 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 18 05:02:17.403817 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 05:02:17.403817 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 05:02:17.403817 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 05:02:17.403817 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 05:02:17.403817 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 05:02:17.403817 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jun 18 05:02:17.403817 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 05:02:17.403817 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jun 18 05:02:17.403817 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 05:02:17.403817 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 05:02:17.403817 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 05:02:17.403817 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jun 18 05:02:17.404824 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 05:02:17.404824 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jun 18 05:02:17.404824 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 18 05:02:17.404824 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 18 05:02:17.404824 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 18 05:02:17.404824 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 18 05:02:17.404824 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 18 05:02:17.404824 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 18 05:02:17.404824 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 18 05:02:17.404824 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 05:02:17.404824 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 637/1253] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Jun 18 05:02:17 2022] GET /identity/v3/users/placement => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 05:02:17.409904 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb04b789-e616-4d17-9b6d-57364d2d2ddc None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:02:17.466739 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:02:17.511214 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb04b789-e616-4d17-9b6d-57364d2d2ddc None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:02:17.546121 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd663b11-054a-4cd3-956f-b99f9e69599d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:02:17.546774 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd663b11-054a-4cd3-956f-b99f9e69599d None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:02:17.547054 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd663b11-054a-4cd3-956f-b99f9e69599d None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:02:17.547328 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd663b11-054a-4cd3-956f-b99f9e69599d None admin] PATH_INFO: `/v3/users` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:02:17.548140 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd663b11-054a-4cd3-956f-b99f9e69599d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 05:02:17.548447 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd663b11-054a-4cd3-956f-b99f9e69599d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:02:17.549892 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd663b11-054a-4cd3-956f-b99f9e69599d None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:02:17.566993 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 617/1254] 10.222.0.23 () {56 vars in 1143 bytes} [Sat Jun 18 05:02:17 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 05:02:17.573021 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fd891d-403d-483f-b2c5-633da6756b6d None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:02:17.672329 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fd891d-403d-483f-b2c5-633da6756b6d None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:02:17.708604 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fc76d9-7a46-4882-85ca-289bc307908d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:02:17.709247 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fc76d9-7a46-4882-85ca-289bc307908d None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:02:17.709571 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fc76d9-7a46-4882-85ca-289bc307908d None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:02:17.709850 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fc76d9-7a46-4882-85ca-289bc307908d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:02:17.765165 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:02:17.777868 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fc76d9-7a46-4882-85ca-289bc307908d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:02:17.779346 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fc76d9-7a46-4882-85ca-289bc307908d None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:02:17.876606 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 638/1255] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Jun 18 05:02:17 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 18 05:02:17.882477 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd663b11-054a-4cd3-956f-b99f9e69599d None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:02:17.986514 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd663b11-054a-4cd3-956f-b99f9e69599d None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:02:17.999917 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:02:18.020706 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aba42ab-fa6b-4e5e-9b08-2fbcaadd79f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:02:18.021519 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aba42ab-fa6b-4e5e-9b08-2fbcaadd79f3 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:02:18.021802 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aba42ab-fa6b-4e5e-9b08-2fbcaadd79f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:02:18.022135 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aba42ab-fa6b-4e5e-9b08-2fbcaadd79f3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:02:18.033443 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aba42ab-fa6b-4e5e-9b08-2fbcaadd79f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:02:18.034923 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aba42ab-fa6b-4e5e-9b08-2fbcaadd79f3 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:02:18.052169 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application [None req-0aba42ab-fa6b-4e5e-9b08-2fbcaadd79f3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 05:02:18.052169 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 05:02:18.052169 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 05:02:18.052169 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 05:02:18.052169 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 05:02:18.052169 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 05:02:18.052169 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 05:02:18.052169 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 05:02:18.052169 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 05:02:18.052169 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 05:02:18.052169 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 05:02:18.052169 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 05:02:18.052169 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 18 05:02:18.052169 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 18 05:02:18.052169 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 18 05:02:18.053211 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 18 05:02:18.053211 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 05:02:18.053211 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 05:02:18.053211 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 05:02:18.053211 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 05:02:18.053211 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 05:02:18.053211 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 05:02:18.053211 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 05:02:18.053211 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application with Lock( Jun 18 05:02:18.053211 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 05:02:18.053211 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._enter() Jun 18 05:02:18.053211 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 05:02:18.053211 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 05:02:18.053211 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 05:02:18.053211 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.creator() Jun 18 05:02:18.053211 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 05:02:18.053211 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application created_value = creator( Jun 18 05:02:18.059103 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 18 05:02:18.059103 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 18 05:02:18.059103 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 05:02:18.059103 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 05:02:18.059103 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 618/1256] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Jun 18 05:02:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 05:02:18.059520 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fc76d9-7a46-4882-85ca-289bc307908d None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:02:18.147140 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:02:18.159915 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fc76d9-7a46-4882-85ca-289bc307908d None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:02:18.193315 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a70e6a-8fef-43ed-b049-ae7b0779aa04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:02:18.194000 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a70e6a-8fef-43ed-b049-ae7b0779aa04 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:02:18.194283 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a70e6a-8fef-43ed-b049-ae7b0779aa04 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:02:18.194563 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a70e6a-8fef-43ed-b049-ae7b0779aa04 None admin] PATH_INFO: `/v3/domains` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:02:18.195272 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a70e6a-8fef-43ed-b049-ae7b0779aa04 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 05:02:18.196082 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a70e6a-8fef-43ed-b049-ae7b0779aa04 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:02:18.197465 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a70e6a-8fef-43ed-b049-ae7b0779aa04 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:02:18.208432 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 639/1257] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Jun 18 05:02:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 05:02:18.214712 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aba42ab-fa6b-4e5e-9b08-2fbcaadd79f3 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:02:18.315824 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aba42ab-fa6b-4e5e-9b08-2fbcaadd79f3 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:02:18.349508 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98330ce3-5bc0-4d36-9e0b-06c39aac3bea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:02:18.350288 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98330ce3-5bc0-4d36-9e0b-06c39aac3bea None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:02:18.350636 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98330ce3-5bc0-4d36-9e0b-06c39aac3bea None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:02:18.350982 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98330ce3-5bc0-4d36-9e0b-06c39aac3bea None admin] PATH_INFO: `/v3/projects/service` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:02:18.357734 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98330ce3-5bc0-4d36-9e0b-06c39aac3bea None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:02:18.359187 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98330ce3-5bc0-4d36-9e0b-06c39aac3bea None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:02:18.373711 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application [None req-98330ce3-5bc0-4d36-9e0b-06c39aac3bea None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 18 05:02:18.373711 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 05:02:18.373711 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 05:02:18.373711 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 05:02:18.373711 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 05:02:18.373711 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 05:02:18.373711 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 05:02:18.373711 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 05:02:18.373711 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 05:02:18.373711 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 05:02:18.373711 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 05:02:18.373711 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 05:02:18.373711 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 18 05:02:18.373711 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 18 05:02:18.373711 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 18 05:02:18.374786 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 18 05:02:18.374786 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 05:02:18.374786 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 05:02:18.374786 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 05:02:18.374786 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 05:02:18.374786 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 05:02:18.374786 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 05:02:18.374786 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 05:02:18.374786 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application with Lock( Jun 18 05:02:18.374786 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 05:02:18.374786 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._enter() Jun 18 05:02:18.374786 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 05:02:18.374786 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 05:02:18.374786 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 05:02:18.374786 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.creator() Jun 18 05:02:18.374786 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 05:02:18.374786 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application created_value = creator( Jun 18 05:02:18.375831 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 18 05:02:18.375831 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 18 05:02:18.375831 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 18 05:02:18.375831 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 18 05:02:18.375831 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 18 05:02:18.375831 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 18 05:02:18.375831 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 18 05:02:18.375831 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 05:02:18.375831 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 619/1258] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Jun 18 05:02:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 05:02:18.381007 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a70e6a-8fef-43ed-b049-ae7b0779aa04 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:02:18.481370 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a70e6a-8fef-43ed-b049-ae7b0779aa04 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:02:18.515328 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b04197-f2bd-487a-b847-704c9f876b8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:02:18.515975 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b04197-f2bd-487a-b847-704c9f876b8e None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:02:18.516262 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b04197-f2bd-487a-b847-704c9f876b8e None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:02:18.516537 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b04197-f2bd-487a-b847-704c9f876b8e None admin] PATH_INFO: `/v3/projects` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:02:18.517317 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b04197-f2bd-487a-b847-704c9f876b8e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 05:02:18.517660 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b04197-f2bd-487a-b847-704c9f876b8e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:02:18.519092 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b04197-f2bd-487a-b847-704c9f876b8e None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:02:18.530826 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 640/1259] 10.222.0.23 () {56 vars in 1148 bytes} [Sat Jun 18 05:02:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 05:02:18.537102 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98330ce3-5bc0-4d36-9e0b-06c39aac3bea None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:02:18.563781 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:02:18.639204 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98330ce3-5bc0-4d36-9e0b-06c39aac3bea None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:02:18.673223 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f9fed4-cc60-498e-9889-b0f38d87a1e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:02:18.673900 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f9fed4-cc60-498e-9889-b0f38d87a1e0 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:02:18.674187 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f9fed4-cc60-498e-9889-b0f38d87a1e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:02:18.674455 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f9fed4-cc60-498e-9889-b0f38d87a1e0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:02:18.675285 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f9fed4-cc60-498e-9889-b0f38d87a1e0 None admin] RBAC: Adding query filter params (role.id=21c65f2c51bd4dd283d13507d2714467, scope.project.id=c95f8f3084c5472cb8364b7c57207394, user.id=5323ffd91f864680b0c0084eb3f44d21) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 05:02:18.675604 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f9fed4-cc60-498e-9889-b0f38d87a1e0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:02:18.677158 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f9fed4-cc60-498e-9889-b0f38d87a1e0 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:02:18.683336 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 620/1260] 10.222.0.23 () {56 vars in 1374 bytes} [Sat Jun 18 05:02:18 2022] GET /identity/v3/role_assignments?user.id=5323ffd91f864680b0c0084eb3f44d21&scope.project.id=c95f8f3084c5472cb8364b7c57207394&role.id=21c65f2c51bd4dd283d13507d2714467 => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 05:02:20.720403 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf561492-491f-4997-a756-0f071252dc3b None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:02:20.720924 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf561492-491f-4997-a756-0f071252dc3b None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:02:20.720924 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf561492-491f-4997-a756-0f071252dc3b None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:02:20.722327 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 641/1261] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 05:02:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 05:02:20.727667 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee589cac-f51a-4acb-8f0a-c340c1252d06 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:02:20.728099 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee589cac-f51a-4acb-8f0a-c340c1252d06 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:02:20.728226 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee589cac-f51a-4acb-8f0a-c340c1252d06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:02:20.745612 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-ee589cac-f51a-4acb-8f0a-c340c1252d06 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:02:20.747050 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-ee589cac-f51a-4acb-8f0a-c340c1252d06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:02:20.815786 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 621/1262] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 05:02:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 18 05:02:20.838911 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf561492-491f-4997-a756-0f071252dc3b None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:02:20.849605 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:02:20.943761 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf561492-491f-4997-a756-0f071252dc3b None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:02:20.979628 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11c90bc-0a4c-4d44-8772-afe48e07b6a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:02:20.980320 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11c90bc-0a4c-4d44-8772-afe48e07b6a9 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:02:20.980601 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11c90bc-0a4c-4d44-8772-afe48e07b6a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:02:20.980871 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11c90bc-0a4c-4d44-8772-afe48e07b6a9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:02:20.990686 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11c90bc-0a4c-4d44-8772-afe48e07b6a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 18 05:02:20.999084 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-c11c90bc-0a4c-4d44-8772-afe48e07b6a9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 18 05:02:20.999084 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 05:02:20.999084 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 05:02:20.999084 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 05:02:20.999084 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 05:02:20.999084 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 05:02:20.999084 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 05:02:20.999084 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 05:02:20.999084 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 05:02:20.999084 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 05:02:20.999084 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 05:02:20.999084 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 05:02:20.999084 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 18 05:02:20.999084 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 18 05:02:20.999084 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 18 05:02:20.999084 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 18 05:02:21.001598 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 18 05:02:21.001598 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 18 05:02:21.001598 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 18 05:02:21.001598 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 18 05:02:21.001598 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 05:02:21.001598 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 05:02:21.001598 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 05:02:21.001598 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 05:02:21.001598 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 05:02:21.001598 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 05:02:21.001598 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 05:02:21.001598 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jun 18 05:02:21.001598 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 05:02:21.001598 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jun 18 05:02:21.001598 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 05:02:21.001598 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 05:02:21.002482 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 05:02:21.002482 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jun 18 05:02:21.002482 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 05:02:21.002482 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jun 18 05:02:21.002482 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 18 05:02:21.002482 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 18 05:02:21.002482 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 18 05:02:21.002482 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 18 05:02:21.002482 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 18 05:02:21.002482 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 18 05:02:21.002482 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 18 05:02:21.002482 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 05:02:21.002482 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 642/1263] 10.222.0.23 () {56 vars in 1096 bytes} [Sat Jun 18 05:02:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 05:02:21.006289 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee589cac-f51a-4acb-8f0a-c340c1252d06 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:02:21.025958 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:02:21.105322 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee589cac-f51a-4acb-8f0a-c340c1252d06 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:02:21.139683 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe98148-15cf-4535-8206-4bac2aa495f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:02:21.140345 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe98148-15cf-4535-8206-4bac2aa495f6 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:02:21.140630 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe98148-15cf-4535-8206-4bac2aa495f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:02:21.140907 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe98148-15cf-4535-8206-4bac2aa495f6 None admin] PATH_INFO: `/v3/roles` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:02:21.141682 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe98148-15cf-4535-8206-4bac2aa495f6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 05:02:21.141996 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe98148-15cf-4535-8206-4bac2aa495f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:02:21.143353 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe98148-15cf-4535-8206-4bac2aa495f6 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:02:21.152095 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 622/1264] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Jun 18 05:02:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 05:02:21.157795 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11c90bc-0a4c-4d44-8772-afe48e07b6a9 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:02:21.223386 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:02:21.256415 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11c90bc-0a4c-4d44-8772-afe48e07b6a9 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:02:21.291835 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328b8fbc-5cde-4232-8f7e-cda660b34e35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:02:21.292609 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328b8fbc-5cde-4232-8f7e-cda660b34e35 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:02:21.292887 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328b8fbc-5cde-4232-8f7e-cda660b34e35 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:02:21.293153 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328b8fbc-5cde-4232-8f7e-cda660b34e35 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:02:21.303346 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328b8fbc-5cde-4232-8f7e-cda660b34e35 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:02:21.305297 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328b8fbc-5cde-4232-8f7e-cda660b34e35 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:02:21.323265 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-328b8fbc-5cde-4232-8f7e-cda660b34e35 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 05:02:21.323265 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 05:02:21.323265 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 05:02:21.323265 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 05:02:21.323265 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 05:02:21.323265 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 05:02:21.323265 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 05:02:21.323265 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 05:02:21.323265 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 05:02:21.323265 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 05:02:21.323265 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 05:02:21.323265 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 05:02:21.323265 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 18 05:02:21.323265 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 18 05:02:21.323265 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 18 05:02:21.324165 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 18 05:02:21.324165 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 05:02:21.324165 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 05:02:21.324165 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 05:02:21.324165 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 05:02:21.324165 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 05:02:21.324165 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 05:02:21.324165 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 05:02:21.324165 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jun 18 05:02:21.324165 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 05:02:21.324165 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jun 18 05:02:21.324165 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 05:02:21.324165 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 05:02:21.324165 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 05:02:21.324165 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jun 18 05:02:21.324165 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 05:02:21.324165 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jun 18 05:02:21.325235 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 18 05:02:21.325235 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 18 05:02:21.325235 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 05:02:21.325235 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 05:02:21.325235 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 643/1265] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Jun 18 05:02:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 05:02:21.330256 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe98148-15cf-4535-8206-4bac2aa495f6 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:02:21.413523 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:02:21.432938 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe98148-15cf-4535-8206-4bac2aa495f6 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:02:21.465938 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a9b7ef-cbcf-460e-a3cb-61d80fac788b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:02:21.466593 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a9b7ef-cbcf-460e-a3cb-61d80fac788b None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:02:21.466882 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a9b7ef-cbcf-460e-a3cb-61d80fac788b None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:02:21.467159 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a9b7ef-cbcf-460e-a3cb-61d80fac788b None admin] PATH_INFO: `/v3/domains` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:02:21.467877 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a9b7ef-cbcf-460e-a3cb-61d80fac788b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 05:02:21.468191 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a9b7ef-cbcf-460e-a3cb-61d80fac788b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:02:21.469586 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a9b7ef-cbcf-460e-a3cb-61d80fac788b None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:02:21.481197 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 623/1266] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Jun 18 05:02:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 05:02:21.486901 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328b8fbc-5cde-4232-8f7e-cda660b34e35 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:02:21.590580 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328b8fbc-5cde-4232-8f7e-cda660b34e35 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:02:21.610372 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:02:21.625354 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bb94a7-d9d0-4d05-8b3c-89c1ba6585e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:02:21.626203 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bb94a7-d9d0-4d05-8b3c-89c1ba6585e7 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:02:21.626666 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bb94a7-d9d0-4d05-8b3c-89c1ba6585e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:02:21.626948 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bb94a7-d9d0-4d05-8b3c-89c1ba6585e7 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:02:21.635444 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bb94a7-d9d0-4d05-8b3c-89c1ba6585e7 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:02:21.637030 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bb94a7-d9d0-4d05-8b3c-89c1ba6585e7 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:02:21.651340 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-68bb94a7-d9d0-4d05-8b3c-89c1ba6585e7 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 18 05:02:21.651340 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 05:02:21.651340 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 05:02:21.651340 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 05:02:21.651340 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 05:02:21.651340 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 05:02:21.651340 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 05:02:21.651340 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 05:02:21.651340 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 05:02:21.651340 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 05:02:21.651340 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 05:02:21.651340 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 05:02:21.651340 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 18 05:02:21.651340 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 18 05:02:21.651340 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 18 05:02:21.651340 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 18 05:02:21.654839 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 05:02:21.654839 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 05:02:21.654839 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 18 05:02:21.654839 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 18 05:02:21.654839 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 18 05:02:21.654839 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 18 05:02:21.654839 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 05:02:21.654839 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 05:02:21.654839 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 05:02:21.654839 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 05:02:21.654839 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 05:02:21.654839 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jun 18 05:02:21.654839 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 05:02:21.654839 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jun 18 05:02:21.654839 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 05:02:21.654839 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 05:02:21.654839 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 05:02:21.654839 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jun 18 05:02:21.655795 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 05:02:21.655795 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jun 18 05:02:21.655795 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 18 05:02:21.655795 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 18 05:02:21.655795 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 18 05:02:21.655795 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 18 05:02:21.655795 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 18 05:02:21.655795 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 18 05:02:21.655795 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 18 05:02:21.655795 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 05:02:21.655795 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 644/1267] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Jun 18 05:02:21 2022] GET /identity/v3/users/placement => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 05:02:21.658298 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a9b7ef-cbcf-460e-a3cb-61d80fac788b None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:02:21.756560 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a9b7ef-cbcf-460e-a3cb-61d80fac788b None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:02:21.791167 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3b48da-62c5-471a-bb30-0fd30b7d574e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:02:21.791824 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3b48da-62c5-471a-bb30-0fd30b7d574e None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:02:21.792104 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3b48da-62c5-471a-bb30-0fd30b7d574e None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:02:21.792371 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3b48da-62c5-471a-bb30-0fd30b7d574e None admin] PATH_INFO: `/v3/users` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:02:21.793184 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3b48da-62c5-471a-bb30-0fd30b7d574e None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 05:02:21.793519 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3b48da-62c5-471a-bb30-0fd30b7d574e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:02:21.794917 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3b48da-62c5-471a-bb30-0fd30b7d574e None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:02:21.803406 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:02:21.812219 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 624/1268] 10.222.0.23 () {56 vars in 1143 bytes} [Sat Jun 18 05:02:21 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 05:02:21.817865 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bb94a7-d9d0-4d05-8b3c-89c1ba6585e7 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:02:21.918168 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bb94a7-d9d0-4d05-8b3c-89c1ba6585e7 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:02:21.950790 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4963fb32-469a-48e2-b384-a72b1dd50d45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:02:21.951567 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4963fb32-469a-48e2-b384-a72b1dd50d45 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:02:21.951851 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4963fb32-469a-48e2-b384-a72b1dd50d45 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:02:21.952127 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4963fb32-469a-48e2-b384-a72b1dd50d45 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:02:21.962503 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4963fb32-469a-48e2-b384-a72b1dd50d45 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:02:21.963956 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4963fb32-469a-48e2-b384-a72b1dd50d45 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:02:21.982344 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-4963fb32-469a-48e2-b384-a72b1dd50d45 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 05:02:21.982344 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 05:02:21.982344 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 05:02:21.982344 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 05:02:21.982344 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 05:02:21.982344 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 05:02:21.982344 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 05:02:21.982344 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 05:02:21.982344 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 05:02:21.982344 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 05:02:21.982344 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 05:02:21.982344 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 05:02:21.982344 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 18 05:02:21.982344 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 18 05:02:21.982344 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 18 05:02:21.983473 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 18 05:02:21.983473 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 05:02:21.983473 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 05:02:21.983473 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 05:02:21.983473 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 05:02:21.983473 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 05:02:21.983473 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 05:02:21.983473 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 05:02:21.983473 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jun 18 05:02:21.983473 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 05:02:21.983473 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jun 18 05:02:21.983473 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 05:02:21.983473 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 05:02:21.983473 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 05:02:21.983473 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jun 18 05:02:21.983473 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 05:02:21.983473 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jun 18 05:02:21.989189 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 18 05:02:21.989189 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 18 05:02:21.989189 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 05:02:21.989189 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 05:02:21.989189 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 645/1269] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Jun 18 05:02:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 05:02:21.989532 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3b48da-62c5-471a-bb30-0fd30b7d574e None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:02:22.088468 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3b48da-62c5-471a-bb30-0fd30b7d574e None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:02:22.122517 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315162d8-1ec9-4828-a20a-0798bc71d188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:02:22.123182 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315162d8-1ec9-4828-a20a-0798bc71d188 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:02:22.123513 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315162d8-1ec9-4828-a20a-0798bc71d188 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:02:22.123807 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315162d8-1ec9-4828-a20a-0798bc71d188 None admin] PATH_INFO: `/v3/domains` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:02:22.124518 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315162d8-1ec9-4828-a20a-0798bc71d188 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 05:02:22.124837 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315162d8-1ec9-4828-a20a-0798bc71d188 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:02:22.126239 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315162d8-1ec9-4828-a20a-0798bc71d188 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:02:22.137935 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 625/1270] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Jun 18 05:02:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 05:02:22.143540 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4963fb32-469a-48e2-b384-a72b1dd50d45 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:02:22.176039 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:02:22.244469 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4963fb32-469a-48e2-b384-a72b1dd50d45 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:02:22.279331 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6b21da-9fea-4e4b-b520-c1223edde2ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:02:22.280115 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6b21da-9fea-4e4b-b520-c1223edde2ef None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:02:22.280399 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6b21da-9fea-4e4b-b520-c1223edde2ef None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:02:22.280674 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6b21da-9fea-4e4b-b520-c1223edde2ef None admin] PATH_INFO: `/v3/projects/service` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:02:22.289070 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6b21da-9fea-4e4b-b520-c1223edde2ef None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:02:22.291079 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6b21da-9fea-4e4b-b520-c1223edde2ef None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:02:22.305082 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-3c6b21da-9fea-4e4b-b520-c1223edde2ef None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 18 05:02:22.305082 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 05:02:22.305082 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 05:02:22.305082 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 05:02:22.305082 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 05:02:22.305082 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 05:02:22.305082 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 05:02:22.305082 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 05:02:22.305082 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 05:02:22.305082 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 05:02:22.305082 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 05:02:22.305082 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 05:02:22.305082 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 18 05:02:22.305082 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 18 05:02:22.305082 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 18 05:02:22.306264 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 18 05:02:22.306264 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 05:02:22.306264 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 05:02:22.306264 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 05:02:22.306264 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 05:02:22.306264 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 05:02:22.306264 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 05:02:22.306264 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 05:02:22.306264 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jun 18 05:02:22.306264 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 05:02:22.306264 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jun 18 05:02:22.306264 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 05:02:22.306264 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 05:02:22.306264 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 05:02:22.306264 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jun 18 05:02:22.306264 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 05:02:22.306264 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jun 18 05:02:22.307626 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 18 05:02:22.307626 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 18 05:02:22.307626 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 18 05:02:22.307626 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 18 05:02:22.307626 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 18 05:02:22.307626 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 18 05:02:22.307626 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 18 05:02:22.307626 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 05:02:22.307626 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 646/1271] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Jun 18 05:02:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 05:02:22.312027 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315162d8-1ec9-4828-a20a-0798bc71d188 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:02:22.370065 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:02:22.413012 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315162d8-1ec9-4828-a20a-0798bc71d188 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:02:22.447059 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a309a1cb-1b1b-47dc-9f6b-5c70ecfb927d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:02:22.447779 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a309a1cb-1b1b-47dc-9f6b-5c70ecfb927d None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:02:22.448085 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a309a1cb-1b1b-47dc-9f6b-5c70ecfb927d None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:02:22.448359 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a309a1cb-1b1b-47dc-9f6b-5c70ecfb927d None admin] PATH_INFO: `/v3/projects` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:02:22.449140 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a309a1cb-1b1b-47dc-9f6b-5c70ecfb927d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 05:02:22.449477 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a309a1cb-1b1b-47dc-9f6b-5c70ecfb927d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:02:22.450895 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a309a1cb-1b1b-47dc-9f6b-5c70ecfb927d None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:02:22.462872 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 626/1272] 10.222.0.23 () {56 vars in 1148 bytes} [Sat Jun 18 05:02:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 05:02:22.468860 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6b21da-9fea-4e4b-b520-c1223edde2ef None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:02:22.561138 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:02:22.569709 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6b21da-9fea-4e4b-b520-c1223edde2ef None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:02:22.604289 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2968f70e-4482-4bb5-b198-d3ced0f99200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:02:22.605163 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2968f70e-4482-4bb5-b198-d3ced0f99200 None admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:02:22.605472 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2968f70e-4482-4bb5-b198-d3ced0f99200 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:02:22.605752 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2968f70e-4482-4bb5-b198-d3ced0f99200 None admin] PATH_INFO: `/v3/projects/c95f8f3084c5472cb8364b7c57207394/users/5323ffd91f864680b0c0084eb3f44d21/roles/21c65f2c51bd4dd283d13507d2714467` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:02:22.607856 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2968f70e-4482-4bb5-b198-d3ced0f99200 None admin] RBAC: Authorizing `identity:create_grant(project_id=c95f8f3084c5472cb8364b7c57207394, user_id=5323ffd91f864680b0c0084eb3f44d21, role_id=21c65f2c51bd4dd283d13507d2714467)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:02:22.609322 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2968f70e-4482-4bb5-b198-d3ced0f99200 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:02:22.695328 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 647/1273] 10.222.0.23 () {58 vars in 1439 bytes} [Sat Jun 18 05:02:22 2022] PUT /identity/v3/projects/c95f8f3084c5472cb8364b7c57207394/users/5323ffd91f864680b0c0084eb3f44d21/roles/21c65f2c51bd4dd283d13507d2714467 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:02:24.705456 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a3159a-d2e2-4f9f-b9ee-ae516f9cfe09 None None] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:02:24.706004 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a3159a-d2e2-4f9f-b9ee-ae516f9cfe09 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:02:24.706004 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a3159a-d2e2-4f9f-b9ee-ae516f9cfe09 None None] PATH_INFO: `/` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:02:24.707351 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 627/1274] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 05:02:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 05:02:24.713370 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52ebf33-965c-4b06-8601-671376d83fb2 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:02:24.713774 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52ebf33-965c-4b06-8601-671376d83fb2 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:02:24.715926 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52ebf33-965c-4b06-8601-671376d83fb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:02:24.730514 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-e52ebf33-965c-4b06-8601-671376d83fb2 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:02:24.731698 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-e52ebf33-965c-4b06-8601-671376d83fb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:02:24.797927 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 648/1275] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 05:02:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 18 05:02:24.821650 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a3159a-d2e2-4f9f-b9ee-ae516f9cfe09 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:02:24.924034 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a3159a-d2e2-4f9f-b9ee-ae516f9cfe09 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:02:24.940977 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:02:24.957938 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9385a224-d5ff-4067-a95b-5f58b54fb606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:02:24.958648 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9385a224-d5ff-4067-a95b-5f58b54fb606 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:02:24.958926 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9385a224-d5ff-4067-a95b-5f58b54fb606 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:02:24.959195 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9385a224-d5ff-4067-a95b-5f58b54fb606 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:02:24.969460 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9385a224-d5ff-4067-a95b-5f58b54fb606 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 18 05:02:24.976827 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application [None req-9385a224-d5ff-4067-a95b-5f58b54fb606 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 18 05:02:24.976827 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 05:02:24.976827 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 05:02:24.976827 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 05:02:24.976827 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 05:02:24.976827 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 05:02:24.976827 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 05:02:24.976827 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 05:02:24.976827 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 05:02:24.976827 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 05:02:24.976827 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 05:02:24.976827 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 05:02:24.976827 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 18 05:02:24.976827 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 18 05:02:24.976827 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 18 05:02:24.976827 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 18 05:02:24.977958 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 18 05:02:24.977958 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 18 05:02:24.977958 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 18 05:02:24.977958 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 18 05:02:24.977958 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 05:02:24.977958 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 05:02:24.977958 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 05:02:24.977958 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 05:02:24.977958 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 05:02:24.977958 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 05:02:24.977958 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 05:02:24.977958 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application with Lock( Jun 18 05:02:24.977958 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 05:02:24.977958 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._enter() Jun 18 05:02:24.977958 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 05:02:24.977958 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 05:02:24.983949 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 05:02:24.983949 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.creator() Jun 18 05:02:24.983949 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 05:02:24.983949 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application created_value = creator( Jun 18 05:02:24.983949 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 18 05:02:24.983949 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 18 05:02:24.983949 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 18 05:02:24.983949 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 18 05:02:24.983949 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 18 05:02:24.983949 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 18 05:02:24.983949 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 18 05:02:24.983949 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 05:02:24.983949 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 628/1276] 10.222.0.23 () {56 vars in 1096 bytes} [Sat Jun 18 05:02:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 05:02:24.984795 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52ebf33-965c-4b06-8601-671376d83fb2 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:02:25.067181 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:02:25.084877 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52ebf33-965c-4b06-8601-671376d83fb2 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:02:25.118985 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2176007-da3a-4458-96c5-b2cc7a4e2a67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:02:25.119740 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2176007-da3a-4458-96c5-b2cc7a4e2a67 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:02:25.120021 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2176007-da3a-4458-96c5-b2cc7a4e2a67 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:02:25.120291 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2176007-da3a-4458-96c5-b2cc7a4e2a67 None admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:02:25.121289 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2176007-da3a-4458-96c5-b2cc7a4e2a67 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 05:02:25.121625 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2176007-da3a-4458-96c5-b2cc7a4e2a67 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:02:25.123069 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2176007-da3a-4458-96c5-b2cc7a4e2a67 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:02:25.131277 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 649/1277] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Jun 18 05:02:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 05:02:25.137047 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9385a224-d5ff-4067-a95b-5f58b54fb606 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:02:25.237905 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9385a224-d5ff-4067-a95b-5f58b54fb606 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:02:25.270424 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab3d49b-805d-44cd-aab2-60dac5114628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:02:25.271200 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab3d49b-805d-44cd-aab2-60dac5114628 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:02:25.271481 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab3d49b-805d-44cd-aab2-60dac5114628 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:02:25.271754 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab3d49b-805d-44cd-aab2-60dac5114628 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:02:25.282494 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab3d49b-805d-44cd-aab2-60dac5114628 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:02:25.283921 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab3d49b-805d-44cd-aab2-60dac5114628 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:02:25.301178 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application [None req-fab3d49b-805d-44cd-aab2-60dac5114628 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 05:02:25.301178 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 05:02:25.301178 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 05:02:25.301178 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 05:02:25.301178 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 05:02:25.301178 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 05:02:25.301178 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 05:02:25.301178 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 05:02:25.301178 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 05:02:25.301178 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 05:02:25.301178 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 05:02:25.301178 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 05:02:25.301178 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 18 05:02:25.301178 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 18 05:02:25.301178 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 18 05:02:25.302249 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 18 05:02:25.302249 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 05:02:25.302249 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 05:02:25.302249 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 05:02:25.302249 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 05:02:25.302249 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 05:02:25.302249 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 05:02:25.302249 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 05:02:25.302249 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application with Lock( Jun 18 05:02:25.302249 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 05:02:25.302249 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._enter() Jun 18 05:02:25.302249 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 05:02:25.302249 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 05:02:25.302249 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 05:02:25.302249 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.creator() Jun 18 05:02:25.302249 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 05:02:25.302249 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application created_value = creator( Jun 18 05:02:25.303712 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 18 05:02:25.303712 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 18 05:02:25.303712 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 05:02:25.303712 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 05:02:25.303712 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 629/1278] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Jun 18 05:02:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 05:02:25.308468 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2176007-da3a-4458-96c5-b2cc7a4e2a67 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:02:25.409760 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2176007-da3a-4458-96c5-b2cc7a4e2a67 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:02:25.442441 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8199964a-c430-4004-a6dd-1e0621118d43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:02:25.443092 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8199964a-c430-4004-a6dd-1e0621118d43 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:02:25.443376 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8199964a-c430-4004-a6dd-1e0621118d43 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:02:25.443657 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8199964a-c430-4004-a6dd-1e0621118d43 None admin] PATH_INFO: `/v3/domains` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:02:25.444369 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8199964a-c430-4004-a6dd-1e0621118d43 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 05:02:25.444681 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8199964a-c430-4004-a6dd-1e0621118d43 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:02:25.446081 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8199964a-c430-4004-a6dd-1e0621118d43 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:02:25.458103 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 650/1279] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Jun 18 05:02:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 18 05:02:25.464992 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab3d49b-805d-44cd-aab2-60dac5114628 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:02:25.499546 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:02:25.563457 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab3d49b-805d-44cd-aab2-60dac5114628 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:02:25.597262 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6076a896-2f02-4d6f-9d16-c56295122cfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:02:25.598078 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6076a896-2f02-4d6f-9d16-c56295122cfd None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:02:25.598363 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6076a896-2f02-4d6f-9d16-c56295122cfd None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:02:25.598637 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6076a896-2f02-4d6f-9d16-c56295122cfd None admin] PATH_INFO: `/v3/users/placement` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:02:25.606376 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6076a896-2f02-4d6f-9d16-c56295122cfd None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:02:25.607878 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6076a896-2f02-4d6f-9d16-c56295122cfd None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:02:25.623136 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application [None req-6076a896-2f02-4d6f-9d16-c56295122cfd None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 18 05:02:25.623136 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 05:02:25.623136 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 05:02:25.623136 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 05:02:25.623136 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 05:02:25.623136 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 05:02:25.623136 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 05:02:25.623136 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 05:02:25.623136 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 05:02:25.623136 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 05:02:25.623136 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 05:02:25.623136 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 05:02:25.623136 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 18 05:02:25.623136 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 18 05:02:25.623136 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 18 05:02:25.623136 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 18 05:02:25.624188 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 05:02:25.624188 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 05:02:25.624188 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 18 05:02:25.624188 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 18 05:02:25.624188 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 18 05:02:25.624188 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 18 05:02:25.624188 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 05:02:25.624188 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 05:02:25.624188 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 05:02:25.624188 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 05:02:25.624188 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 05:02:25.624188 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application with Lock( Jun 18 05:02:25.624188 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 05:02:25.624188 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._enter() Jun 18 05:02:25.624188 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 05:02:25.624188 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 05:02:25.624188 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 05:02:25.624188 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.creator() Jun 18 05:02:25.625166 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 05:02:25.625166 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application created_value = creator( Jun 18 05:02:25.625166 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 18 05:02:25.625166 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 18 05:02:25.625166 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 18 05:02:25.625166 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 18 05:02:25.625166 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 18 05:02:25.625166 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 18 05:02:25.625166 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 18 05:02:25.625166 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 05:02:25.625166 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 630/1280] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Jun 18 05:02:25 2022] GET /identity/v3/users/placement => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 05:02:25.631051 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8199964a-c430-4004-a6dd-1e0621118d43 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:02:25.635482 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:02:25.729681 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8199964a-c430-4004-a6dd-1e0621118d43 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:02:25.764141 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c1de9d-c444-4bba-9610-9ade6b498dc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:02:25.764726 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c1de9d-c444-4bba-9610-9ade6b498dc2 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:02:25.765005 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c1de9d-c444-4bba-9610-9ade6b498dc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:02:25.765277 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c1de9d-c444-4bba-9610-9ade6b498dc2 None admin] PATH_INFO: `/v3/users` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:02:25.766182 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c1de9d-c444-4bba-9610-9ade6b498dc2 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 05:02:25.766488 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c1de9d-c444-4bba-9610-9ade6b498dc2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:02:25.767901 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c1de9d-c444-4bba-9610-9ade6b498dc2 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:02:25.787011 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 651/1281] 10.222.0.23 () {56 vars in 1143 bytes} [Sat Jun 18 05:02:25 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 05:02:25.793621 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6076a896-2f02-4d6f-9d16-c56295122cfd None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:02:25.890279 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:02:25.900910 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6076a896-2f02-4d6f-9d16-c56295122cfd None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:02:25.936460 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84455a02-c395-4ca1-b65c-93a5c529a751 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:02:25.938075 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84455a02-c395-4ca1-b65c-93a5c529a751 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:02:25.938075 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84455a02-c395-4ca1-b65c-93a5c529a751 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:02:25.938075 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84455a02-c395-4ca1-b65c-93a5c529a751 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:02:26.013169 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84455a02-c395-4ca1-b65c-93a5c529a751 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:02:26.014731 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84455a02-c395-4ca1-b65c-93a5c529a751 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:02:26.116060 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:02:26.131972 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 631/1282] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Jun 18 05:02:25 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 18 05:02:26.133239 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c1de9d-c444-4bba-9610-9ade6b498dc2 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:02:26.199865 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:02:26.238553 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c1de9d-c444-4bba-9610-9ade6b498dc2 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:02:26.275903 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7167ca76-cd94-4a6c-ad0a-e1e20518e3e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:02:26.276644 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7167ca76-cd94-4a6c-ad0a-e1e20518e3e6 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:02:26.276929 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7167ca76-cd94-4a6c-ad0a-e1e20518e3e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:02:26.277202 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7167ca76-cd94-4a6c-ad0a-e1e20518e3e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:02:26.287821 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7167ca76-cd94-4a6c-ad0a-e1e20518e3e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:02:26.289288 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7167ca76-cd94-4a6c-ad0a-e1e20518e3e6 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:02:26.307112 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-7167ca76-cd94-4a6c-ad0a-e1e20518e3e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 05:02:26.307112 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 05:02:26.307112 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 05:02:26.307112 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 05:02:26.307112 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 05:02:26.307112 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 05:02:26.307112 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 05:02:26.307112 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 05:02:26.307112 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 05:02:26.307112 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 05:02:26.307112 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 05:02:26.307112 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 05:02:26.307112 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 18 05:02:26.307112 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 18 05:02:26.307112 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 18 05:02:26.308200 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 18 05:02:26.308200 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 05:02:26.308200 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 05:02:26.308200 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 05:02:26.308200 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 05:02:26.308200 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 05:02:26.308200 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 05:02:26.308200 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 05:02:26.308200 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jun 18 05:02:26.308200 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 05:02:26.308200 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jun 18 05:02:26.308200 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 05:02:26.308200 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 05:02:26.308200 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 05:02:26.308200 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jun 18 05:02:26.308200 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 05:02:26.308200 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jun 18 05:02:26.309306 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 18 05:02:26.309306 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 18 05:02:26.309306 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 18 05:02:26.309306 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 05:02:26.309306 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 652/1283] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Jun 18 05:02:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 05:02:26.314246 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84455a02-c395-4ca1-b65c-93a5c529a751 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:02:26.418220 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84455a02-c395-4ca1-b65c-93a5c529a751 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:02:26.452967 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b016dd-4cd9-41c2-b6eb-2ec50171a755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:02:26.453656 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b016dd-4cd9-41c2-b6eb-2ec50171a755 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:02:26.453940 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b016dd-4cd9-41c2-b6eb-2ec50171a755 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:02:26.454216 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b016dd-4cd9-41c2-b6eb-2ec50171a755 None admin] PATH_INFO: `/v3/domains` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:02:26.454926 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b016dd-4cd9-41c2-b6eb-2ec50171a755 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 05:02:26.455234 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b016dd-4cd9-41c2-b6eb-2ec50171a755 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:02:26.456599 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b016dd-4cd9-41c2-b6eb-2ec50171a755 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:02:26.468317 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 632/1284] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Jun 18 05:02:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 05:02:26.474225 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7167ca76-cd94-4a6c-ad0a-e1e20518e3e6 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:02:26.575560 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7167ca76-cd94-4a6c-ad0a-e1e20518e3e6 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:02:26.590285 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:02:26.609833 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70b4db0-3ac7-4c5b-9bbc-d0b7a17f8ed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:02:26.610640 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70b4db0-3ac7-4c5b-9bbc-d0b7a17f8ed1 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:02:26.610919 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70b4db0-3ac7-4c5b-9bbc-d0b7a17f8ed1 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:02:26.611190 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70b4db0-3ac7-4c5b-9bbc-d0b7a17f8ed1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:02:26.617697 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70b4db0-3ac7-4c5b-9bbc-d0b7a17f8ed1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:02:26.619899 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70b4db0-3ac7-4c5b-9bbc-d0b7a17f8ed1 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:02:26.633329 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-e70b4db0-3ac7-4c5b-9bbc-d0b7a17f8ed1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 18 05:02:26.633329 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 05:02:26.633329 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 05:02:26.633329 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 05:02:26.633329 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 05:02:26.633329 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 05:02:26.633329 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 05:02:26.633329 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 05:02:26.633329 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 05:02:26.633329 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 05:02:26.633329 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 05:02:26.633329 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 05:02:26.633329 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 18 05:02:26.633329 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 18 05:02:26.633329 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 18 05:02:26.635179 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 18 05:02:26.635179 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 05:02:26.635179 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 05:02:26.635179 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 05:02:26.635179 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 05:02:26.635179 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 05:02:26.635179 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 05:02:26.635179 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 05:02:26.635179 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jun 18 05:02:26.635179 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 05:02:26.635179 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jun 18 05:02:26.635179 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 05:02:26.635179 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 05:02:26.635179 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 05:02:26.635179 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jun 18 05:02:26.635179 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 05:02:26.635179 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jun 18 05:02:26.636393 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 18 05:02:26.636393 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 18 05:02:26.636393 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 18 05:02:26.636393 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 18 05:02:26.636393 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 18 05:02:26.636393 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 18 05:02:26.636393 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 18 05:02:26.636393 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jun 18 05:02:26.636393 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 653/1285] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Jun 18 05:02:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 18 05:02:26.641192 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b016dd-4cd9-41c2-b6eb-2ec50171a755 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:02:26.690799 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:02:26.739966 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b016dd-4cd9-41c2-b6eb-2ec50171a755 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:02:26.773979 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45ac260-df74-41cf-b6eb-ddc4625757eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:02:26.774634 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45ac260-df74-41cf-b6eb-ddc4625757eb None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:02:26.775013 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45ac260-df74-41cf-b6eb-ddc4625757eb None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:02:26.775423 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45ac260-df74-41cf-b6eb-ddc4625757eb None admin] PATH_INFO: `/v3/projects` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:02:26.776205 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45ac260-df74-41cf-b6eb-ddc4625757eb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 05:02:26.776517 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45ac260-df74-41cf-b6eb-ddc4625757eb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:02:26.777978 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45ac260-df74-41cf-b6eb-ddc4625757eb None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:02:26.789853 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 633/1286] 10.222.0.23 () {56 vars in 1148 bytes} [Sat Jun 18 05:02:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 05:02:26.796271 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70b4db0-3ac7-4c5b-9bbc-d0b7a17f8ed1 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:02:26.896770 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70b4db0-3ac7-4c5b-9bbc-d0b7a17f8ed1 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:02:26.929826 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afba934-f13d-41ea-974e-b0ced3257ddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:02:26.930469 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afba934-f13d-41ea-974e-b0ced3257ddd None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:02:26.930747 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afba934-f13d-41ea-974e-b0ced3257ddd None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:02:26.931015 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afba934-f13d-41ea-974e-b0ced3257ddd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:02:26.932899 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afba934-f13d-41ea-974e-b0ced3257ddd None admin] RBAC: Adding query filter params (role.id=21c65f2c51bd4dd283d13507d2714467, scope.project.id=c95f8f3084c5472cb8364b7c57207394, user.id=5323ffd91f864680b0c0084eb3f44d21) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 05:02:26.933213 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afba934-f13d-41ea-974e-b0ced3257ddd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:02:26.934717 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afba934-f13d-41ea-974e-b0ced3257ddd None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:02:26.940422 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 654/1287] 10.222.0.23 () {56 vars in 1374 bytes} [Sat Jun 18 05:02:26 2022] GET /identity/v3/role_assignments?user.id=5323ffd91f864680b0c0084eb3f44d21&scope.project.id=c95f8f3084c5472cb8364b7c57207394&role.id=21c65f2c51bd4dd283d13507d2714467 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 05:02:29.004750 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73af115d-8f0c-4fce-89bf-beaa65ed86cc None None] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:02:29.005277 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73af115d-8f0c-4fce-89bf-beaa65ed86cc None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:02:29.005422 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73af115d-8f0c-4fce-89bf-beaa65ed86cc None None] PATH_INFO: `/` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:02:29.006722 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 634/1288] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 05:02:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 05:02:29.013080 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4ff9fd-8e19-4b34-b84a-f2e3f337e8f3 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:02:29.013598 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4ff9fd-8e19-4b34-b84a-f2e3f337e8f3 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:02:29.013708 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4ff9fd-8e19-4b34-b84a-f2e3f337e8f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:02:29.030195 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-1d4ff9fd-8e19-4b34-b84a-f2e3f337e8f3 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:02:29.031394 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-1d4ff9fd-8e19-4b34-b84a-f2e3f337e8f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:02:29.060876 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:02:29.097340 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 655/1289] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 05:02:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 18 05:02:29.121310 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73af115d-8f0c-4fce-89bf-beaa65ed86cc None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:02:29.221405 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73af115d-8f0c-4fce-89bf-beaa65ed86cc None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:02:29.232682 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:02:29.257093 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ee181f-8d0a-4d29-a4d8-3e5085c9c22a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:02:29.257858 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ee181f-8d0a-4d29-a4d8-3e5085c9c22a None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:02:29.258144 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ee181f-8d0a-4d29-a4d8-3e5085c9c22a None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:02:29.258421 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ee181f-8d0a-4d29-a4d8-3e5085c9c22a None admin] PATH_INFO: `/v3/services/placement` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:02:29.259069 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ee181f-8d0a-4d29-a4d8-3e5085c9c22a None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:02:29.260503 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ee181f-8d0a-4d29-a4d8-3e5085c9c22a None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:02:29.271942 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application [None req-f6ee181f-8d0a-4d29-a4d8-3e5085c9c22a None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 18 05:02:29.271942 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 05:02:29.271942 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 18 05:02:29.271942 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 18 05:02:29.271942 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 18 05:02:29.271942 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 18 05:02:29.271942 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 18 05:02:29.271942 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 18 05:02:29.271942 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 18 05:02:29.271942 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 05:02:29.271942 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 18 05:02:29.271942 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 05:02:29.271942 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 05:02:29.271942 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 05:02:29.271942 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 05:02:29.271942 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 05:02:29.271942 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 05:02:29.273373 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 05:02:29.273373 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 05:02:29.273373 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 05:02:29.273373 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 05:02:29.273373 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 05:02:29.273373 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 05:02:29.273373 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 18 05:02:29.273373 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 18 05:02:29.273373 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 18 05:02:29.273373 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 18 05:02:29.273373 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 05:02:29.273373 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 05:02:29.273373 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 05:02:29.273373 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 05:02:29.273373 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 05:02:29.273373 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 05:02:29.274518 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 05:02:29.274518 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application with Lock( Jun 18 05:02:29.274518 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 05:02:29.274518 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._enter() Jun 18 05:02:29.274518 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 05:02:29.274518 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 05:02:29.274518 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 05:02:29.274518 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.creator() Jun 18 05:02:29.274518 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 05:02:29.274518 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application created_value = creator( Jun 18 05:02:29.274518 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 18 05:02:29.274518 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 18 05:02:29.274518 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 18 05:02:29.274518 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 05:02:29.274518 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 635/1290] 10.222.0.23 () {56 vars in 1117 bytes} [Sat Jun 18 05:02:29 2022] GET /identity/v3/services/placement => generated 90 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 05:02:29.279180 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4ff9fd-8e19-4b34-b84a-f2e3f337e8f3 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:02:29.375877 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4ff9fd-8e19-4b34-b84a-f2e3f337e8f3 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:02:29.410769 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716cc02a-7ea2-4080-801c-019699db9e6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:02:29.411429 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716cc02a-7ea2-4080-801c-019699db9e6b None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:02:29.411714 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716cc02a-7ea2-4080-801c-019699db9e6b None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:02:29.411991 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716cc02a-7ea2-4080-801c-019699db9e6b None admin] PATH_INFO: `/v3/services` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:02:29.412692 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716cc02a-7ea2-4080-801c-019699db9e6b None admin] RBAC: Adding query filter params (name=placement) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 05:02:29.413005 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716cc02a-7ea2-4080-801c-019699db9e6b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:02:29.414407 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716cc02a-7ea2-4080-801c-019699db9e6b None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:02:29.419513 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 656/1291] 10.222.0.23 () {56 vars in 1116 bytes} [Sat Jun 18 05:02:29 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 05:02:29.426410 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ee181f-8d0a-4d29-a4d8-3e5085c9c22a None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:02:29.526669 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ee181f-8d0a-4d29-a4d8-3e5085c9c22a None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:02:29.560198 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b60069-79e5-4b25-b225-a5f8cf61caf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:02:29.560838 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b60069-79e5-4b25-b225-a5f8cf61caf7 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:02:29.561118 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b60069-79e5-4b25-b225-a5f8cf61caf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:02:29.561413 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b60069-79e5-4b25-b225-a5f8cf61caf7 None admin] PATH_INFO: `/v3/services` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:02:29.562110 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b60069-79e5-4b25-b225-a5f8cf61caf7 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 05:02:29.562492 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b60069-79e5-4b25-b225-a5f8cf61caf7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:02:29.563838 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b60069-79e5-4b25-b225-a5f8cf61caf7 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:02:29.568311 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 636/1292] 10.222.0.23 () {56 vars in 1116 bytes} [Sat Jun 18 05:02:29 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 18 05:02:31.573780 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbfcc69-df6b-4e71-924b-334490d22af3 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:02:31.574325 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbfcc69-df6b-4e71-924b-334490d22af3 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:02:31.574325 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbfcc69-df6b-4e71-924b-334490d22af3 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:02:31.575679 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 657/1293] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 05:02:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 05:02:31.580880 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2411d332-5f16-492e-ac3d-30a278518a04 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:02:31.581306 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2411d332-5f16-492e-ac3d-30a278518a04 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:02:31.581549 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2411d332-5f16-492e-ac3d-30a278518a04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:02:31.598569 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-2411d332-5f16-492e-ac3d-30a278518a04 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:02:31.599747 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-2411d332-5f16-492e-ac3d-30a278518a04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:02:31.628745 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:02:31.666028 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 637/1294] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 05:02:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 18 05:02:31.688097 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbfcc69-df6b-4e71-924b-334490d22af3 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:02:31.719632 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:02:31.788350 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbfcc69-df6b-4e71-924b-334490d22af3 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:02:31.821823 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2453434e-15a9-4156-b064-c405b146a1f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:02:31.822910 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2453434e-15a9-4156-b064-c405b146a1f6 None admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:02:31.823193 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2453434e-15a9-4156-b064-c405b146a1f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:02:31.823468 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2453434e-15a9-4156-b064-c405b146a1f6 None admin] PATH_INFO: `/v3/services` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:02:31.824238 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2453434e-15a9-4156-b064-c405b146a1f6 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:02:31.825683 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2453434e-15a9-4156-b064-c405b146a1f6 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:02:31.918523 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 658/1295] 10.222.0.23 () {60 vars in 1141 bytes} [Sat Jun 18 05:02:31 2022] POST /identity/v3/services => generated 246 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:02:33.968862 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899ad91c-2a92-4591-8981-9701d2f15655 None None] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:02:33.969428 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899ad91c-2a92-4591-8981-9701d2f15655 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:02:33.969428 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899ad91c-2a92-4591-8981-9701d2f15655 None None] PATH_INFO: `/` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:02:33.970799 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 638/1296] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 05:02:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 05:02:33.975853 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881e6ca4-ecc9-4d89-8d37-fd226ac01675 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:02:33.976288 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881e6ca4-ecc9-4d89-8d37-fd226ac01675 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:02:33.976406 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881e6ca4-ecc9-4d89-8d37-fd226ac01675 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:02:33.993069 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-881e6ca4-ecc9-4d89-8d37-fd226ac01675 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:02:33.995145 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-881e6ca4-ecc9-4d89-8d37-fd226ac01675 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:02:34.066641 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 659/1297] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 05:02:33 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 18 05:02:34.090711 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899ad91c-2a92-4591-8981-9701d2f15655 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:02:34.191819 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899ad91c-2a92-4591-8981-9701d2f15655 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:02:34.226153 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add35be8-6ed0-4060-8eb0-13b94823a98b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:02:34.226870 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add35be8-6ed0-4060-8eb0-13b94823a98b None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:02:34.227152 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add35be8-6ed0-4060-8eb0-13b94823a98b None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:02:34.227424 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add35be8-6ed0-4060-8eb0-13b94823a98b None admin] PATH_INFO: `/v3/services/placement` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:02:34.228112 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add35be8-6ed0-4060-8eb0-13b94823a98b None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:02:34.229455 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add35be8-6ed0-4060-8eb0-13b94823a98b None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:02:34.240918 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application [None req-add35be8-6ed0-4060-8eb0-13b94823a98b None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 18 05:02:34.240918 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 05:02:34.240918 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 18 05:02:34.240918 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 18 05:02:34.240918 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 18 05:02:34.240918 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 18 05:02:34.240918 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 18 05:02:34.240918 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 18 05:02:34.240918 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 18 05:02:34.240918 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 05:02:34.240918 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 18 05:02:34.240918 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 05:02:34.240918 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 05:02:34.240918 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 05:02:34.240918 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 05:02:34.240918 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 05:02:34.240918 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 05:02:34.242121 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 05:02:34.242121 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 05:02:34.242121 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 05:02:34.242121 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 05:02:34.242121 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 05:02:34.242121 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 05:02:34.242121 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 18 05:02:34.242121 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 18 05:02:34.242121 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 18 05:02:34.242121 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 18 05:02:34.242121 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 05:02:34.242121 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 05:02:34.242121 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 05:02:34.242121 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 05:02:34.242121 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 05:02:34.242121 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 05:02:34.243030 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 05:02:34.243030 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application with Lock( Jun 18 05:02:34.243030 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 05:02:34.243030 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._enter() Jun 18 05:02:34.243030 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 05:02:34.243030 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 05:02:34.243030 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 05:02:34.243030 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.creator() Jun 18 05:02:34.243030 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 05:02:34.243030 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application created_value = creator( Jun 18 05:02:34.243030 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 18 05:02:34.243030 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 18 05:02:34.243030 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 18 05:02:34.243030 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 05:02:34.243030 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 639/1298] 10.222.0.23 () {56 vars in 1117 bytes} [Sat Jun 18 05:02:34 2022] GET /identity/v3/services/placement => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 05:02:34.249001 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881e6ca4-ecc9-4d89-8d37-fd226ac01675 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:02:34.267702 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:02:34.347229 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881e6ca4-ecc9-4d89-8d37-fd226ac01675 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:02:34.383318 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41a2776-2d7b-4f45-80ab-d75e95e7d574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:02:34.383974 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41a2776-2d7b-4f45-80ab-d75e95e7d574 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:02:34.384318 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41a2776-2d7b-4f45-80ab-d75e95e7d574 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:02:34.385198 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41a2776-2d7b-4f45-80ab-d75e95e7d574 None admin] PATH_INFO: `/v3/services` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:02:34.385980 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41a2776-2d7b-4f45-80ab-d75e95e7d574 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 05:02:34.386289 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41a2776-2d7b-4f45-80ab-d75e95e7d574 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:02:34.387666 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41a2776-2d7b-4f45-80ab-d75e95e7d574 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:02:34.392997 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 660/1299] 10.222.0.23 () {56 vars in 1116 bytes} [Sat Jun 18 05:02:34 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 05:02:34.398930 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add35be8-6ed0-4060-8eb0-13b94823a98b None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:02:34.432129 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:02:34.500905 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add35be8-6ed0-4060-8eb0-13b94823a98b None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:02:34.532733 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd8cd71-d9fa-4754-b37f-3aec8011281a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:02:34.533454 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd8cd71-d9fa-4754-b37f-3aec8011281a None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:02:34.538406 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd8cd71-d9fa-4754-b37f-3aec8011281a None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:02:34.538406 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd8cd71-d9fa-4754-b37f-3aec8011281a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:02:34.538406 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd8cd71-d9fa-4754-b37f-3aec8011281a None admin] RBAC: Adding query filter params (interface=public, service_id=f2544c1548644ecaaeb5cf5185ffd6bd, region_id=RegionOne) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 05:02:34.538406 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd8cd71-d9fa-4754-b37f-3aec8011281a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:02:34.538406 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd8cd71-d9fa-4754-b37f-3aec8011281a None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:02:34.542312 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 640/1300] 10.222.0.23 () {56 vars in 1251 bytes} [Sat Jun 18 05:02:34 2022] GET /identity/v3/endpoints?service_id=f2544c1548644ecaaeb5cf5185ffd6bd&interface=public®ion_id=RegionOne => generated 194 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 05:02:34.548247 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41a2776-2d7b-4f45-80ab-d75e95e7d574 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:02:34.630361 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:02:34.647884 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41a2776-2d7b-4f45-80ab-d75e95e7d574 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:02:34.682526 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebd30cc-841f-465d-b548-8732a5fd12b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:02:34.683253 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebd30cc-841f-465d-b548-8732a5fd12b4 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:02:34.683531 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebd30cc-841f-465d-b548-8732a5fd12b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:02:34.683808 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebd30cc-841f-465d-b548-8732a5fd12b4 None admin] PATH_INFO: `/v3/services` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:02:34.684483 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebd30cc-841f-465d-b548-8732a5fd12b4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:02:34.685884 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebd30cc-841f-465d-b548-8732a5fd12b4 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:02:34.692713 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 661/1301] 10.222.0.23 () {56 vars in 1087 bytes} [Sat Jun 18 05:02:34 2022] GET /identity/v3/services => generated 2191 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:02:36.753726 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e9d14a-b733-4635-8a52-c48e43ed034c None None] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:02:36.754281 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e9d14a-b733-4635-8a52-c48e43ed034c None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:02:36.754281 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e9d14a-b733-4635-8a52-c48e43ed034c None None] PATH_INFO: `/` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:02:36.755635 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 641/1302] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 05:02:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 05:02:36.761375 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd96878-496e-4dcf-8023-f4e63e97bcb9 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:02:36.761988 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd96878-496e-4dcf-8023-f4e63e97bcb9 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:02:36.761988 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd96878-496e-4dcf-8023-f4e63e97bcb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:02:36.779113 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-8dd96878-496e-4dcf-8023-f4e63e97bcb9 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:02:36.780545 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-8dd96878-496e-4dcf-8023-f4e63e97bcb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:02:36.875106 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 662/1303] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 05:02:36 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 18 05:02:36.900485 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e9d14a-b733-4635-8a52-c48e43ed034c None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:02:36.999490 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:02:37.006639 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e9d14a-b733-4635-8a52-c48e43ed034c None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:02:37.042290 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56573a00-0d86-4421-91e9-9d505c430810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:02:37.043118 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56573a00-0d86-4421-91e9-9d505c430810 None admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:02:37.043645 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56573a00-0d86-4421-91e9-9d505c430810 None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:02:37.044102 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56573a00-0d86-4421-91e9-9d505c430810 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:02:37.045126 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56573a00-0d86-4421-91e9-9d505c430810 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:02:37.046535 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56573a00-0d86-4421-91e9-9d505c430810 None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:02:37.060318 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application [None req-56573a00-0d86-4421-91e9-9d505c430810 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 18 05:02:37.060318 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 05:02:37.060318 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 18 05:02:37.060318 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 18 05:02:37.060318 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 18 05:02:37.060318 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 18 05:02:37.060318 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 18 05:02:37.060318 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 18 05:02:37.060318 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 18 05:02:37.060318 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 05:02:37.060318 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 18 05:02:37.060318 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 05:02:37.060318 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 18 05:02:37.060318 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 18 05:02:37.060318 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 18 05:02:37.060318 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 18 05:02:37.060318 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 18 05:02:37.061352 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 18 05:02:37.061352 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 18 05:02:37.061352 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 18 05:02:37.061352 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 18 05:02:37.061352 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 18 05:02:37.061352 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 18 05:02:37.061352 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 18 05:02:37.061352 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 18 05:02:37.061352 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 18 05:02:37.061352 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 18 05:02:37.061352 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 18 05:02:37.061352 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 18 05:02:37.061352 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 18 05:02:37.061352 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 18 05:02:37.061352 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 18 05:02:37.061352 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.get_or_create( Jun 18 05:02:37.062273 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 18 05:02:37.062273 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application with Lock( Jun 18 05:02:37.062273 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 18 05:02:37.062273 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self._enter() Jun 18 05:02:37.062273 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 18 05:02:37.062273 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 18 05:02:37.062273 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 18 05:02:37.062273 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application return self.creator() Jun 18 05:02:37.062273 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 18 05:02:37.062273 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application created_value = creator( Jun 18 05:02:37.062273 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 18 05:02:37.062273 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 18 05:02:37.062273 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 18 05:02:37.062273 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: ERROR keystone.server.flask.application Jun 18 05:02:37.062273 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 642/1304] 10.222.0.23 () {56 vars in 1117 bytes} [Sat Jun 18 05:02:36 2022] GET /identity/v3/services/placement => generated 90 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 18 05:02:37.067805 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd96878-496e-4dcf-8023-f4e63e97bcb9 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:02:37.156725 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:02:37.180993 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd96878-496e-4dcf-8023-f4e63e97bcb9 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:02:37.220484 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83b69ce-bcae-4f88-b028-da629762242d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:02:37.221134 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83b69ce-bcae-4f88-b028-da629762242d None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:02:37.221586 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83b69ce-bcae-4f88-b028-da629762242d None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:02:37.221876 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83b69ce-bcae-4f88-b028-da629762242d None admin] PATH_INFO: `/v3/services` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:02:37.222672 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83b69ce-bcae-4f88-b028-da629762242d None admin] RBAC: Adding query filter params (name=placement) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 05:02:37.222989 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83b69ce-bcae-4f88-b028-da629762242d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:02:37.224356 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83b69ce-bcae-4f88-b028-da629762242d None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:02:37.231711 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 663/1305] 10.222.0.23 () {56 vars in 1116 bytes} [Sat Jun 18 05:02:37 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 05:02:37.237602 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56573a00-0d86-4421-91e9-9d505c430810 None admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:02:37.341300 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56573a00-0d86-4421-91e9-9d505c430810 None admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:02:37.375994 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075d5761-12d1-49ca-8f4c-7fcb953c9a8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:02:37.376631 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075d5761-12d1-49ca-8f4c-7fcb953c9a8b None admin] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:02:37.376907 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075d5761-12d1-49ca-8f4c-7fcb953c9a8b None admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:02:37.377185 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075d5761-12d1-49ca-8f4c-7fcb953c9a8b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:02:37.377967 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075d5761-12d1-49ca-8f4c-7fcb953c9a8b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:02:37.379329 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075d5761-12d1-49ca-8f4c-7fcb953c9a8b None admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:02:37.467131 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 643/1306] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Jun 18 05:02:37 2022] POST /identity/v3/endpoints => generated 331 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:02:47.226582 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=30 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:02:47.462366 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:08:25.051843 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc4da39-99ca-464d-a8f4-e491fb3489a1 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:08:25.052901 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc4da39-99ca-464d-a8f4-e491fb3489a1 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:08:25.053467 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc4da39-99ca-464d-a8f4-e491fb3489a1 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:08:25.055099 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 664/1307] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 05:08:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 05:08:25.060979 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bfe28b-189a-43c2-9db0-92718b4aecc4 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:08:25.061468 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bfe28b-189a-43c2-9db0-92718b4aecc4 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:08:25.061552 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bfe28b-189a-43c2-9db0-92718b4aecc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:08:25.225749 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-32bfe28b-189a-43c2-9db0-92718b4aecc4 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:08:25.227321 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-32bfe28b-189a-43c2-9db0-92718b4aecc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:08:25.457525 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 644/1308] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 05:08:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 388 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:08:25.507797 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3fcaf2-85b5-477c-b6fb-c09a8223b442 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:08:25.508399 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3fcaf2-85b5-477c-b6fb-c09a8223b442 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:08:25.509447 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3fcaf2-85b5-477c-b6fb-c09a8223b442 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:08:25.511890 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 665/1309] 10.222.0.23 () {54 vars in 964 bytes} [Sat Jun 18 05:08:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 05:08:25.526123 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafac012-9e64-4226-9aae-ba4c268277b0 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:08:25.526507 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafac012-9e64-4226-9aae-ba4c268277b0 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:08:25.526864 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafac012-9e64-4226-9aae-ba4c268277b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:08:25.612396 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-bafac012-9e64-4226-9aae-ba4c268277b0 None None] MFA Rules not processed for user `e78c4d7cde8f4ad9aa479bb09e325db3`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:08:25.613823 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-bafac012-9e64-4226-9aae-ba4c268277b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:08:25.638629 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:08:25.704097 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 645/1310] 10.222.0.23 () {58 vars in 1061 bytes} [Sat Jun 18 05:08:25 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:08:25.714563 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3fcaf2-85b5-477c-b6fb-c09a8223b442 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:08:25.716824 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-2d3fcaf2-85b5-477c-b6fb-c09a8223b442 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:08:25.840036 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3fcaf2-85b5-477c-b6fb-c09a8223b442 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:08:25.926430 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb46e15e-9ca3-4f05-b88f-8f88d861b0e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c4d7cde8f4ad9aa479bb09e325db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:08:25.927111 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb46e15e-9ca3-4f05-b88f-8f88d861b0e6 service neutron] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:08:25.927396 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb46e15e-9ca3-4f05-b88f-8f88d861b0e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:08:25.927672 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb46e15e-9ca3-4f05-b88f-8f88d861b0e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:08:25.929637 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-bb46e15e-9ca3-4f05-b88f-8f88d861b0e6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:08:25.997908 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:08:26.064671 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb46e15e-9ca3-4f05-b88f-8f88d861b0e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:08:26.066513 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb46e15e-9ca3-4f05-b88f-8f88d861b0e6 service neutron] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:08:26.088098 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 666/1311] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:08:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:08:28.463130 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb540d12-fdcf-41cc-9286-15bc1dd10347 None None] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:08:28.463649 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb540d12-fdcf-41cc-9286-15bc1dd10347 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:08:28.463649 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb540d12-fdcf-41cc-9286-15bc1dd10347 None None] PATH_INFO: `/` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:08:28.465176 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 646/1312] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 05:08:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 05:08:28.470381 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257f4de5-9442-4b5d-a848-0ef48e9731b4 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:08:28.470808 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257f4de5-9442-4b5d-a848-0ef48e9731b4 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:08:28.471004 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257f4de5-9442-4b5d-a848-0ef48e9731b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:08:28.544589 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-257f4de5-9442-4b5d-a848-0ef48e9731b4 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:08:28.546011 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-257f4de5-9442-4b5d-a848-0ef48e9731b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:08:28.647737 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 667/1313] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 05:08:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:08:28.686911 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb540d12-fdcf-41cc-9286-15bc1dd10347 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:08:28.688537 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-bb540d12-fdcf-41cc-9286-15bc1dd10347 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:08:28.801454 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb540d12-fdcf-41cc-9286-15bc1dd10347 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:08:28.872650 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:08:28.875188 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3747fa60-f97f-4a01-b347-f5d5494b2180 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c4d7cde8f4ad9aa479bb09e325db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:08:28.875822 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3747fa60-f97f-4a01-b347-f5d5494b2180 service neutron] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:08:28.876102 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3747fa60-f97f-4a01-b347-f5d5494b2180 service neutron] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:08:28.876373 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3747fa60-f97f-4a01-b347-f5d5494b2180 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:08:28.878015 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-3747fa60-f97f-4a01-b347-f5d5494b2180 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:08:28.988531 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3747fa60-f97f-4a01-b347-f5d5494b2180 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:08:28.990010 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3747fa60-f97f-4a01-b347-f5d5494b2180 service neutron] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:08:29.010051 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 647/1314] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:08:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:08:31.591826 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ebaad3-8f4e-4989-861d-9d8f8d9169d1 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:08:31.592404 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ebaad3-8f4e-4989-861d-9d8f8d9169d1 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:08:31.592404 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ebaad3-8f4e-4989-861d-9d8f8d9169d1 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:08:31.593840 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 668/1315] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 05:08:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 05:08:31.598967 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaa5563-8b12-46e7-a327-770c7572730f None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:08:31.599375 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaa5563-8b12-46e7-a327-770c7572730f None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:08:31.599518 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaa5563-8b12-46e7-a327-770c7572730f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:08:31.664461 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-ceaa5563-8b12-46e7-a327-770c7572730f None None] MFA Rules not processed for user `21ca63be55244b159bcc68b518233c91`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:08:31.666355 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-ceaa5563-8b12-46e7-a327-770c7572730f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:08:31.718058 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:08:31.763471 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 648/1316] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 05:08:31 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:08:31.818297 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee6f6de-5eca-4f79-ac84-54b3fe7ceca2 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:08:31.818576 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee6f6de-5eca-4f79-ac84-54b3fe7ceca2 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:08:31.818849 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee6f6de-5eca-4f79-ac84-54b3fe7ceca2 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:08:31.820227 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 669/1317] 10.222.0.23 () {54 vars in 964 bytes} [Sat Jun 18 05:08:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 05:08:31.826044 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fbfa3a-9efc-4684-aaa6-45ac0feddc14 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:08:31.826336 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fbfa3a-9efc-4684-aaa6-45ac0feddc14 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:08:31.826612 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fbfa3a-9efc-4684-aaa6-45ac0feddc14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:08:31.900832 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-e5fbfa3a-9efc-4684-aaa6-45ac0feddc14 None None] MFA Rules not processed for user `e78c4d7cde8f4ad9aa479bb09e325db3`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:08:31.902060 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-e5fbfa3a-9efc-4684-aaa6-45ac0feddc14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:08:31.978026 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 649/1318] 10.222.0.23 () {58 vars in 1061 bytes} [Sat Jun 18 05:08:31 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:08:31.987434 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee6f6de-5eca-4f79-ac84-54b3fe7ceca2 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:08:31.988940 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-fee6f6de-5eca-4f79-ac84-54b3fe7ceca2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:08:31.999307 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:08:32.093627 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee6f6de-5eca-4f79-ac84-54b3fe7ceca2 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:08:32.167456 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f23576b-e2d6-47c2-a364-943a8764b5c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c4d7cde8f4ad9aa479bb09e325db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:08:32.168087 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f23576b-e2d6-47c2-a364-943a8764b5c5 service neutron] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:08:32.168364 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f23576b-e2d6-47c2-a364-943a8764b5c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:08:32.168630 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f23576b-e2d6-47c2-a364-943a8764b5c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:08:32.170280 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-6f23576b-e2d6-47c2-a364-943a8764b5c5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:08:32.232435 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:08:32.279310 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f23576b-e2d6-47c2-a364-943a8764b5c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:08:32.280964 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f23576b-e2d6-47c2-a364-943a8764b5c5 service neutron] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:08:32.300394 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 670/1319] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:08:31 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:08:35.497499 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6201fa-e4bc-411a-b491-eaee824d0ae0 None None] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:08:35.498020 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6201fa-e4bc-411a-b491-eaee824d0ae0 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:08:35.498117 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6201fa-e4bc-411a-b491-eaee824d0ae0 None None] PATH_INFO: `/` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:08:35.499421 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 650/1320] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 05:08:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 05:08:35.504554 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5edb2c5-189c-4b58-8c76-4a5e2a9741dc None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:08:35.504984 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5edb2c5-189c-4b58-8c76-4a5e2a9741dc None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:08:35.505145 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5edb2c5-189c-4b58-8c76-4a5e2a9741dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:08:35.568947 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-d5edb2c5-189c-4b58-8c76-4a5e2a9741dc None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:08:35.570186 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-d5edb2c5-189c-4b58-8c76-4a5e2a9741dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:08:35.664974 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 671/1321] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 05:08:35 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:08:35.704618 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6201fa-e4bc-411a-b491-eaee824d0ae0 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:08:35.706201 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-cc6201fa-e4bc-411a-b491-eaee824d0ae0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:08:35.754420 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:08:35.807126 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6201fa-e4bc-411a-b491-eaee824d0ae0 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:08:35.877767 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d62fcdd-1b62-4af7-bc39-556b8778522a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c4d7cde8f4ad9aa479bb09e325db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:08:35.878396 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d62fcdd-1b62-4af7-bc39-556b8778522a service neutron] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:08:35.878677 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d62fcdd-1b62-4af7-bc39-556b8778522a service neutron] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:08:35.878951 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d62fcdd-1b62-4af7-bc39-556b8778522a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:08:35.880547 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-3d62fcdd-1b62-4af7-bc39-556b8778522a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:08:35.974616 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:08:35.988039 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d62fcdd-1b62-4af7-bc39-556b8778522a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:08:35.989483 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d62fcdd-1b62-4af7-bc39-556b8778522a service neutron] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:08:36.006811 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 651/1322] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:08:35 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:08:39.166509 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c534dbe1-421a-4e31-b417-0d991ef64942 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:08:39.167057 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c534dbe1-421a-4e31-b417-0d991ef64942 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:08:39.167057 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c534dbe1-421a-4e31-b417-0d991ef64942 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:08:39.168435 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 672/1323] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 05:08:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 05:08:39.173563 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7373318-84c9-4172-aa80-e80ea4e0be46 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:08:39.173949 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7373318-84c9-4172-aa80-e80ea4e0be46 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:08:39.174244 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7373318-84c9-4172-aa80-e80ea4e0be46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:08:39.238650 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-b7373318-84c9-4172-aa80-e80ea4e0be46 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:08:39.239847 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-b7373318-84c9-4172-aa80-e80ea4e0be46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:08:39.338311 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 652/1324] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 05:08:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:08:39.373997 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c534dbe1-421a-4e31-b417-0d991ef64942 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:08:39.375547 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-c534dbe1-421a-4e31-b417-0d991ef64942 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:08:39.396356 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:08:39.491609 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c534dbe1-421a-4e31-b417-0d991ef64942 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:08:39.572493 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b223e408-3e13-4023-a2c0-a750a0a55fc8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c4d7cde8f4ad9aa479bb09e325db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:08:39.573142 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b223e408-3e13-4023-a2c0-a750a0a55fc8 service neutron] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:08:39.573493 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b223e408-3e13-4023-a2c0-a750a0a55fc8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:08:39.573810 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b223e408-3e13-4023-a2c0-a750a0a55fc8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:08:39.575523 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-b223e408-3e13-4023-a2c0-a750a0a55fc8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:08:39.651331 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:08:39.688537 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b223e408-3e13-4023-a2c0-a750a0a55fc8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:08:39.690029 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b223e408-3e13-4023-a2c0-a750a0a55fc8 service neutron] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:08:39.708804 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 673/1325] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:08:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:08:41.811399 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a312bc31-7409-499c-a2f9-acbc13085236 None None] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:08:41.811956 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a312bc31-7409-499c-a2f9-acbc13085236 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:08:41.811956 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a312bc31-7409-499c-a2f9-acbc13085236 None None] PATH_INFO: `/` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:08:41.813354 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 653/1326] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 05:08:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 05:08:41.818530 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649bc09e-c448-4092-8edb-ebae9a2a06ed None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:08:41.818934 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649bc09e-c448-4092-8edb-ebae9a2a06ed None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:08:41.819073 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649bc09e-c448-4092-8edb-ebae9a2a06ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:08:41.884741 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-649bc09e-c448-4092-8edb-ebae9a2a06ed None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:08:41.885970 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-649bc09e-c448-4092-8edb-ebae9a2a06ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:08:41.981909 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 674/1327] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 05:08:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:08:42.017944 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a312bc31-7409-499c-a2f9-acbc13085236 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:08:42.019468 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-a312bc31-7409-499c-a2f9-acbc13085236 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:08:42.080428 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:08:42.123958 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a312bc31-7409-499c-a2f9-acbc13085236 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:08:42.198325 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30946c0-8b2b-4849-9287-f2dcca28019a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c4d7cde8f4ad9aa479bb09e325db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:08:42.198969 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30946c0-8b2b-4849-9287-f2dcca28019a service neutron] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:08:42.199250 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30946c0-8b2b-4849-9287-f2dcca28019a service neutron] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:08:42.199520 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30946c0-8b2b-4849-9287-f2dcca28019a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:08:42.201135 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-a30946c0-8b2b-4849-9287-f2dcca28019a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:08:42.314708 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:08:42.317158 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30946c0-8b2b-4849-9287-f2dcca28019a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:08:42.318653 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30946c0-8b2b-4849-9287-f2dcca28019a service neutron] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:08:42.340340 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 654/1328] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:08:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:08:44.449025 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483d0065-0cfd-46fd-ac65-e9482209694e None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:08:44.449554 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483d0065-0cfd-46fd-ac65-e9482209694e None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:08:44.449646 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483d0065-0cfd-46fd-ac65-e9482209694e None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:08:44.450990 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 675/1329] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 05:08:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 05:08:44.456249 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af92d9d9-bbd8-4b87-a789-66505913fac9 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:08:44.456666 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af92d9d9-bbd8-4b87-a789-66505913fac9 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:08:44.456830 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af92d9d9-bbd8-4b87-a789-66505913fac9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:08:44.526480 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-af92d9d9-bbd8-4b87-a789-66505913fac9 None None] MFA Rules not processed for user `21ca63be55244b159bcc68b518233c91`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:08:44.527740 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-af92d9d9-bbd8-4b87-a789-66505913fac9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:08:44.626499 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 655/1330] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 05:08:44 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:08:44.664657 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483d0065-0cfd-46fd-ac65-e9482209694e None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:08:44.666216 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-483d0065-0cfd-46fd-ac65-e9482209694e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:08:44.699538 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:08:44.774396 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483d0065-0cfd-46fd-ac65-e9482209694e None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:08:44.851149 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af053f34-8126-4a73-9c08-709155ba103f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c4d7cde8f4ad9aa479bb09e325db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:08:44.851790 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af053f34-8126-4a73-9c08-709155ba103f service neutron] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:08:44.852071 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af053f34-8126-4a73-9c08-709155ba103f service neutron] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:08:44.852711 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af053f34-8126-4a73-9c08-709155ba103f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:08:44.854031 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-af053f34-8126-4a73-9c08-709155ba103f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:08:44.931470 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:08:44.963463 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af053f34-8126-4a73-9c08-709155ba103f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:08:44.964907 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af053f34-8126-4a73-9c08-709155ba103f service neutron] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:08:44.983498 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 676/1331] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:08:44 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:08:47.444279 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31228093-1b09-4431-be5a-f3dec378f273 None None] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:08:47.444800 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31228093-1b09-4431-be5a-f3dec378f273 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:08:47.444934 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31228093-1b09-4431-be5a-f3dec378f273 None None] PATH_INFO: `/` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:08:47.446793 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 656/1332] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 05:08:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 05:08:47.451878 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a5b90d-d6c9-4c54-a83e-dceb7ef93dce None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:08:47.452304 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a5b90d-d6c9-4c54-a83e-dceb7ef93dce None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:08:47.452435 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a5b90d-d6c9-4c54-a83e-dceb7ef93dce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:08:47.517151 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-22a5b90d-d6c9-4c54-a83e-dceb7ef93dce None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:08:47.518312 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-22a5b90d-d6c9-4c54-a83e-dceb7ef93dce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:08:47.616095 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 677/1333] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 05:08:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:08:47.652485 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31228093-1b09-4431-be5a-f3dec378f273 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:08:47.654030 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-31228093-1b09-4431-be5a-f3dec378f273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:08:47.729066 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:08:47.763783 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31228093-1b09-4431-be5a-f3dec378f273 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:08:47.836150 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ace8c8-0ecd-4926-8316-040ad706ef20 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c4d7cde8f4ad9aa479bb09e325db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:08:47.837631 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ace8c8-0ecd-4926-8316-040ad706ef20 service neutron] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:08:47.837927 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ace8c8-0ecd-4926-8316-040ad706ef20 service neutron] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:08:47.838202 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ace8c8-0ecd-4926-8316-040ad706ef20 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:08:47.839829 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-92ace8c8-0ecd-4926-8316-040ad706ef20 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:08:47.949748 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ace8c8-0ecd-4926-8316-040ad706ef20 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:08:47.951229 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ace8c8-0ecd-4926-8316-040ad706ef20 service neutron] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:08:47.961232 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:08:47.969591 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 657/1334] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:08:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:08:50.063326 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae10901-9c5d-487a-ad07-72878c13d6dd None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:08:50.063972 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae10901-9c5d-487a-ad07-72878c13d6dd None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:08:50.063972 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae10901-9c5d-487a-ad07-72878c13d6dd None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:08:50.065239 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 678/1335] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 05:08:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 05:08:50.070302 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020068dc-c50a-4908-bb28-be1b7b68c6dc None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:08:50.070756 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020068dc-c50a-4908-bb28-be1b7b68c6dc None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:08:50.070895 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020068dc-c50a-4908-bb28-be1b7b68c6dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:08:50.137186 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-020068dc-c50a-4908-bb28-be1b7b68c6dc None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:08:50.138419 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-020068dc-c50a-4908-bb28-be1b7b68c6dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:08:50.234056 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 658/1336] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 05:08:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:08:50.274844 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae10901-9c5d-487a-ad07-72878c13d6dd None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:08:50.276364 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-4ae10901-9c5d-487a-ad07-72878c13d6dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:08:50.315838 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:08:50.378731 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae10901-9c5d-487a-ad07-72878c13d6dd None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:08:50.452082 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5732abd-4d50-4b30-b892-ee4508e58e95 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c4d7cde8f4ad9aa479bb09e325db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:08:50.452716 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5732abd-4d50-4b30-b892-ee4508e58e95 service neutron] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:08:50.453031 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5732abd-4d50-4b30-b892-ee4508e58e95 service neutron] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:08:50.453306 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5732abd-4d50-4b30-b892-ee4508e58e95 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:08:50.454960 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-c5732abd-4d50-4b30-b892-ee4508e58e95 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:08:50.537188 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:08:50.563220 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5732abd-4d50-4b30-b892-ee4508e58e95 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:08:50.564653 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5732abd-4d50-4b30-b892-ee4508e58e95 service neutron] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:08:50.583323 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 679/1337] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:08:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:08:53.317084 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bc54d6-b89f-4dcb-9896-d3d03e312e64 None None] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:08:53.317641 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bc54d6-b89f-4dcb-9896-d3d03e312e64 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:08:53.317641 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bc54d6-b89f-4dcb-9896-d3d03e312e64 None None] PATH_INFO: `/` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:08:53.318969 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 659/1338] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 05:08:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 05:08:53.324954 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09e14d9-47b3-4a42-8030-cbafb53fc58c None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:08:53.325360 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09e14d9-47b3-4a42-8030-cbafb53fc58c None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:08:53.325541 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09e14d9-47b3-4a42-8030-cbafb53fc58c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:08:53.390804 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-b09e14d9-47b3-4a42-8030-cbafb53fc58c None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:08:53.392306 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-b09e14d9-47b3-4a42-8030-cbafb53fc58c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:08:53.488356 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 680/1339] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 05:08:53 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:08:53.525988 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bc54d6-b89f-4dcb-9896-d3d03e312e64 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:08:53.527487 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-11bc54d6-b89f-4dcb-9896-d3d03e312e64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:08:53.605467 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:08:53.632985 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bc54d6-b89f-4dcb-9896-d3d03e312e64 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:08:53.706768 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877cc50c-e094-40c3-a01f-1a63b9279e1b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c4d7cde8f4ad9aa479bb09e325db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:08:53.707409 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877cc50c-e094-40c3-a01f-1a63b9279e1b service neutron] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:08:53.707690 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877cc50c-e094-40c3-a01f-1a63b9279e1b service neutron] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:08:53.707962 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877cc50c-e094-40c3-a01f-1a63b9279e1b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:08:53.709616 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-877cc50c-e094-40c3-a01f-1a63b9279e1b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:08:53.816874 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877cc50c-e094-40c3-a01f-1a63b9279e1b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:08:53.818341 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877cc50c-e094-40c3-a01f-1a63b9279e1b service neutron] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:08:53.837398 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 660/1340] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:08:53 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:08:58.066134 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5bdaa6-c181-4464-9353-40f13aee5ec5 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:08:58.066659 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5bdaa6-c181-4464-9353-40f13aee5ec5 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:08:58.066762 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5bdaa6-c181-4464-9353-40f13aee5ec5 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:08:58.068055 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 681/1341] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 05:08:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 05:08:58.073436 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4492966b-47e9-4ec7-90ef-515bbae59245 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:08:58.073853 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4492966b-47e9-4ec7-90ef-515bbae59245 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:08:58.074132 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4492966b-47e9-4ec7-90ef-515bbae59245 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:08:58.080365 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:08:58.152405 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-4492966b-47e9-4ec7-90ef-515bbae59245 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:08:58.153698 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-4492966b-47e9-4ec7-90ef-515bbae59245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:08:58.266893 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 661/1342] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 05:08:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:08:58.306224 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5bdaa6-c181-4464-9353-40f13aee5ec5 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:08:58.308403 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-fb5bdaa6-c181-4464-9353-40f13aee5ec5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:08:58.359480 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:08:58.411746 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5bdaa6-c181-4464-9353-40f13aee5ec5 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:08:58.416368 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67814ebc-f9f6-4aff-a57b-a0921bfb8cda service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c4d7cde8f4ad9aa479bb09e325db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:08:58.417001 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67814ebc-f9f6-4aff-a57b-a0921bfb8cda service neutron] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:08:58.417283 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67814ebc-f9f6-4aff-a57b-a0921bfb8cda service neutron] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:08:58.417597 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67814ebc-f9f6-4aff-a57b-a0921bfb8cda service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:08:58.426865 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67814ebc-f9f6-4aff-a57b-a0921bfb8cda service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:08:58.428289 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67814ebc-f9f6-4aff-a57b-a0921bfb8cda service neutron] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:08:58.440944 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 682/1343] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:08:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:09:01.368662 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8e6b58-f5ed-4de5-b8f5-35dc529ec4d7 None None] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:09:01.369216 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8e6b58-f5ed-4de5-b8f5-35dc529ec4d7 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:09:01.369572 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8e6b58-f5ed-4de5-b8f5-35dc529ec4d7 None None] PATH_INFO: `/` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:09:01.371126 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 662/1344] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 05:09:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 05:09:01.377022 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85fe356-820e-4e0c-a612-4d2e51fb6ff5 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:09:01.377568 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85fe356-820e-4e0c-a612-4d2e51fb6ff5 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:09:01.377680 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85fe356-820e-4e0c-a612-4d2e51fb6ff5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:09:01.409944 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-e85fe356-820e-4e0c-a612-4d2e51fb6ff5 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:09:01.411190 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-e85fe356-820e-4e0c-a612-4d2e51fb6ff5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:09:01.499216 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 683/1345] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 05:09:01 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:09:01.537836 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8e6b58-f5ed-4de5-b8f5-35dc529ec4d7 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:09:01.549755 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8e6b58-f5ed-4de5-b8f5-35dc529ec4d7 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:09:01.553325 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e499f9-7878-4527-8d5d-bca87efdaa58 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c4d7cde8f4ad9aa479bb09e325db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:09:01.554067 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e499f9-7878-4527-8d5d-bca87efdaa58 service neutron] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:09:01.554422 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e499f9-7878-4527-8d5d-bca87efdaa58 service neutron] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:09:01.554704 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e499f9-7878-4527-8d5d-bca87efdaa58 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:09:01.564536 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e499f9-7878-4527-8d5d-bca87efdaa58 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:09:01.566005 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e499f9-7878-4527-8d5d-bca87efdaa58 service neutron] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:09:01.575822 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 663/1346] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:09:01 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:09:05.828825 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9691542e-0ef5-4753-996d-963f66fe99a4 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:09:05.829293 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9691542e-0ef5-4753-996d-963f66fe99a4 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:09:05.829374 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9691542e-0ef5-4753-996d-963f66fe99a4 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:09:05.831137 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 684/1347] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 05:09:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 05:09:05.836061 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc1cfc0-c4bf-4a27-8ce7-674510b16130 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:09:05.836485 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc1cfc0-c4bf-4a27-8ce7-674510b16130 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:09:05.836639 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc1cfc0-c4bf-4a27-8ce7-674510b16130 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:09:05.854738 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-fcc1cfc0-c4bf-4a27-8ce7-674510b16130 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:09:05.855950 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-fcc1cfc0-c4bf-4a27-8ce7-674510b16130 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:09:05.864088 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 664/1348] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 05:09:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:09:05.903440 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9691542e-0ef5-4753-996d-963f66fe99a4 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:09:05.914062 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9691542e-0ef5-4753-996d-963f66fe99a4 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:09:05.917883 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e9af40-5bf5-4412-a41d-0a12476bdbdd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c4d7cde8f4ad9aa479bb09e325db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:09:05.918516 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e9af40-5bf5-4412-a41d-0a12476bdbdd service neutron] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:09:05.919752 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e9af40-5bf5-4412-a41d-0a12476bdbdd service neutron] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:09:05.919752 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e9af40-5bf5-4412-a41d-0a12476bdbdd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:09:05.928453 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e9af40-5bf5-4412-a41d-0a12476bdbdd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:09:05.929928 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e9af40-5bf5-4412-a41d-0a12476bdbdd service neutron] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:09:05.942559 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 685/1349] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:09:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:09:10.928989 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e77f0c9-f59a-482a-baa7-12e2e7af6fd4 None None] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:09:10.929519 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e77f0c9-f59a-482a-baa7-12e2e7af6fd4 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:09:10.929602 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e77f0c9-f59a-482a-baa7-12e2e7af6fd4 None None] PATH_INFO: `/` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:09:10.934976 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 665/1350] 10.222.0.23 () {54 vars in 921 bytes} [Sat Jun 18 05:09:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 05:09:10.938160 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d2e38a-a0f2-4a7a-aa8e-274770b9dc2f None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:09:10.938593 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d2e38a-a0f2-4a7a-aa8e-274770b9dc2f None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:09:10.938752 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d2e38a-a0f2-4a7a-aa8e-274770b9dc2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:09:11.043573 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-85d2e38a-a0f2-4a7a-aa8e-274770b9dc2f None None] MFA Rules not processed for user `5323ffd91f864680b0c0084eb3f44d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:09:11.045015 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-85d2e38a-a0f2-4a7a-aa8e-274770b9dc2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:09:11.079227 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:09:11.188114 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 686/1351] 10.222.0.23 () {58 vars in 1018 bytes} [Sat Jun 18 05:09:10 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:09:15.444242 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2969ce96-b814-459f-9b0e-f6178e4807d4 None None] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:09:15.444758 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2969ce96-b814-459f-9b0e-f6178e4807d4 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:09:15.445212 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2969ce96-b814-459f-9b0e-f6178e4807d4 None None] PATH_INFO: `/` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:09:15.446746 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 666/1352] 10.222.0.23 () {54 vars in 921 bytes} [Sat Jun 18 05:09:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 05:09:15.452492 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfcaadd-dfcb-4caa-970f-082541361bb6 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:09:15.453004 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfcaadd-dfcb-4caa-970f-082541361bb6 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:09:15.453376 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfcaadd-dfcb-4caa-970f-082541361bb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:09:15.473304 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-2dfcaadd-dfcb-4caa-970f-082541361bb6 None None] MFA Rules not processed for user `5323ffd91f864680b0c0084eb3f44d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:09:15.474665 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-2dfcaadd-dfcb-4caa-970f-082541361bb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:09:15.484072 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 687/1353] 10.222.0.23 () {58 vars in 1018 bytes} [Sat Jun 18 05:09:15 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:09:15.850491 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=72 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:09:17.015319 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db2176b-6002-4280-b64c-edcdc1a1e5c4 None None] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:09:17.015941 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db2176b-6002-4280-b64c-edcdc1a1e5c4 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:09:17.015941 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db2176b-6002-4280-b64c-edcdc1a1e5c4 None None] PATH_INFO: `/` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:09:17.017251 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 667/1354] 10.222.0.23 () {54 vars in 921 bytes} [Sat Jun 18 05:09:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 05:09:17.022155 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceb7045-04df-463b-8f68-fd60c4695906 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:09:17.022699 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceb7045-04df-463b-8f68-fd60c4695906 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:09:17.022976 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceb7045-04df-463b-8f68-fd60c4695906 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:09:17.041608 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-1ceb7045-04df-463b-8f68-fd60c4695906 None None] MFA Rules not processed for user `5323ffd91f864680b0c0084eb3f44d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:09:17.042817 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-1ceb7045-04df-463b-8f68-fd60c4695906 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:09:17.051279 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 688/1355] 10.222.0.23 () {58 vars in 1018 bytes} [Sat Jun 18 05:09:17 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 18 05:09:20.895407 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504e9ff9-2592-4df8-bc04-d81e53ddec0d None None] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:09:20.896285 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504e9ff9-2592-4df8-bc04-d81e53ddec0d None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:09:20.896720 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504e9ff9-2592-4df8-bc04-d81e53ddec0d None None] PATH_INFO: `/` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:09:20.898532 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 668/1356] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 05:09:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 05:09:20.903926 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07ba1eb-8d27-43cc-b60f-efaed1836303 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:09:20.904385 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07ba1eb-8d27-43cc-b60f-efaed1836303 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:09:20.904975 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07ba1eb-8d27-43cc-b60f-efaed1836303 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:09:20.924169 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-f07ba1eb-8d27-43cc-b60f-efaed1836303 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:09:20.925680 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-f07ba1eb-8d27-43cc-b60f-efaed1836303 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:09:20.945404 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 689/1357] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 05:09:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 18 05:09:20.994563 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a3e475-310b-44a2-a5e9-f93ff632f564 None None] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:09:20.995289 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a3e475-310b-44a2-a5e9-f93ff632f564 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:09:20.995793 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a3e475-310b-44a2-a5e9-f93ff632f564 None None] PATH_INFO: `/` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:09:20.998319 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 669/1358] 10.222.0.23 () {54 vars in 961 bytes} [Sat Jun 18 05:09:20 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 05:09:21.007246 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ceaf24-4175-49a5-b54a-b1ff9de49a01 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:09:21.012002 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ceaf24-4175-49a5-b54a-b1ff9de49a01 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:09:21.012002 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ceaf24-4175-49a5-b54a-b1ff9de49a01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:09:21.060991 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-77ceaf24-4175-49a5-b54a-b1ff9de49a01 None None] MFA Rules not processed for user `15e7f45ee0744c1d9602187eac8e8b80`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:09:21.062565 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-77ceaf24-4175-49a5-b54a-b1ff9de49a01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:09:21.161124 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 690/1359] 10.222.0.23 () {58 vars in 1058 bytes} [Sat Jun 18 05:09:21 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 18 05:09:21.167178 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a3e475-310b-44a2-a5e9-f93ff632f564 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:09:21.183283 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a3e475-310b-44a2-a5e9-f93ff632f564 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:09:21.190854 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3137fa6-00d7-4320-ab83-5c16e09a109e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15e7f45ee0744c1d9602187eac8e8b80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:09:21.191824 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3137fa6-00d7-4320-ab83-5c16e09a109e service nova] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:09:21.192336 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3137fa6-00d7-4320-ab83-5c16e09a109e service nova] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:09:21.192792 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3137fa6-00d7-4320-ab83-5c16e09a109e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:09:21.204296 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3137fa6-00d7-4320-ab83-5c16e09a109e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:09:21.206587 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3137fa6-00d7-4320-ab83-5c16e09a109e service nova] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:09:21.218028 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 670/1360] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:09:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:09:23.495615 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5feaa4d-bfdf-4f60-8f92-559f67408297 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:09:23.496180 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5feaa4d-bfdf-4f60-8f92-559f67408297 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:09:23.496180 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5feaa4d-bfdf-4f60-8f92-559f67408297 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:09:23.497753 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 691/1361] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 05:09:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 05:09:23.502746 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbea7f8-b861-4167-ac69-f480b69b7f64 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:09:23.503522 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbea7f8-b861-4167-ac69-f480b69b7f64 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:09:23.503979 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbea7f8-b861-4167-ac69-f480b69b7f64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:09:23.522343 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-6cbea7f8-b861-4167-ac69-f480b69b7f64 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:09:23.523556 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-6cbea7f8-b861-4167-ac69-f480b69b7f64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:09:23.532319 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 671/1362] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 05:09:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:09:23.577136 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5feaa4d-bfdf-4f60-8f92-559f67408297 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:09:23.597764 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5feaa4d-bfdf-4f60-8f92-559f67408297 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:09:23.603951 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac18f8c4-b6dd-4009-a39b-c69af480557f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15e7f45ee0744c1d9602187eac8e8b80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:09:23.604652 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac18f8c4-b6dd-4009-a39b-c69af480557f service nova] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:09:23.604878 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac18f8c4-b6dd-4009-a39b-c69af480557f service nova] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:09:23.605158 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac18f8c4-b6dd-4009-a39b-c69af480557f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:09:23.614167 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac18f8c4-b6dd-4009-a39b-c69af480557f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:09:23.615785 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac18f8c4-b6dd-4009-a39b-c69af480557f service nova] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:09:23.627136 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 692/1363] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:09:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:09:26.206809 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2de0c5-8499-441f-a381-db0feab47c11 None None] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:09:26.207354 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2de0c5-8499-441f-a381-db0feab47c11 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:09:26.207451 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2de0c5-8499-441f-a381-db0feab47c11 None None] PATH_INFO: `/` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:09:26.208923 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 672/1364] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 05:09:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 05:09:26.214891 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58d73fb-b01d-4ff5-9a19-7d144078c26e None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:09:26.215280 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58d73fb-b01d-4ff5-9a19-7d144078c26e None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:09:26.215441 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58d73fb-b01d-4ff5-9a19-7d144078c26e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:09:26.235442 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-a58d73fb-b01d-4ff5-9a19-7d144078c26e None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:09:26.236624 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-a58d73fb-b01d-4ff5-9a19-7d144078c26e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:09:26.245292 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 693/1365] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 05:09:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:09:26.293542 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f9fc9a-3259-4662-958d-30bcefc24f3a None None] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:09:26.293837 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f9fc9a-3259-4662-958d-30bcefc24f3a None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:09:26.294118 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f9fc9a-3259-4662-958d-30bcefc24f3a None None] PATH_INFO: `/` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:09:26.295504 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 673/1366] 10.222.0.23 () {54 vars in 961 bytes} [Sat Jun 18 05:09:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 05:09:26.300851 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0407933c-71c4-41c0-ae9f-598d4311be83 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:09:26.301302 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0407933c-71c4-41c0-ae9f-598d4311be83 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:09:26.301626 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0407933c-71c4-41c0-ae9f-598d4311be83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:09:26.320027 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-0407933c-71c4-41c0-ae9f-598d4311be83 None None] MFA Rules not processed for user `15e7f45ee0744c1d9602187eac8e8b80`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:09:26.322417 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-0407933c-71c4-41c0-ae9f-598d4311be83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:09:26.332215 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 694/1367] 10.222.0.23 () {58 vars in 1058 bytes} [Sat Jun 18 05:09:26 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 18 05:09:26.340827 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f9fc9a-3259-4662-958d-30bcefc24f3a None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:09:26.360398 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f9fc9a-3259-4662-958d-30bcefc24f3a None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:09:26.365973 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d38f0f-a5fd-449c-a72a-cec5b45195ff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15e7f45ee0744c1d9602187eac8e8b80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:09:26.366609 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d38f0f-a5fd-449c-a72a-cec5b45195ff service nova] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:09:26.366894 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d38f0f-a5fd-449c-a72a-cec5b45195ff service nova] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:09:26.367171 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d38f0f-a5fd-449c-a72a-cec5b45195ff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:09:26.376699 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d38f0f-a5fd-449c-a72a-cec5b45195ff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:09:26.377441 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d38f0f-a5fd-449c-a72a-cec5b45195ff service nova] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:09:26.392179 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 674/1368] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:09:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:09:28.636261 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d3a6b7-2eb7-4706-9a8d-f9a2585aa38f None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:09:28.636825 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d3a6b7-2eb7-4706-9a8d-f9a2585aa38f None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:09:28.637153 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d3a6b7-2eb7-4706-9a8d-f9a2585aa38f None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:09:28.638873 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 695/1369] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 05:09:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 05:09:28.644323 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dbb38a-b7df-49ca-8575-8299fc856839 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:09:28.645095 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dbb38a-b7df-49ca-8575-8299fc856839 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:09:28.645621 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dbb38a-b7df-49ca-8575-8299fc856839 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:09:28.664376 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-f7dbb38a-b7df-49ca-8575-8299fc856839 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:09:28.666514 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-f7dbb38a-b7df-49ca-8575-8299fc856839 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:09:28.674799 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 675/1370] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 05:09:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:09:28.718695 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d3a6b7-2eb7-4706-9a8d-f9a2585aa38f None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:09:28.733983 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d3a6b7-2eb7-4706-9a8d-f9a2585aa38f None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:09:28.739772 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babb188e-fbe0-4e21-af4f-d1e82b547df5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15e7f45ee0744c1d9602187eac8e8b80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:09:28.740570 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babb188e-fbe0-4e21-af4f-d1e82b547df5 service nova] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:09:28.740957 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babb188e-fbe0-4e21-af4f-d1e82b547df5 service nova] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:09:28.741334 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babb188e-fbe0-4e21-af4f-d1e82b547df5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:09:28.750781 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babb188e-fbe0-4e21-af4f-d1e82b547df5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:09:28.752537 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babb188e-fbe0-4e21-af4f-d1e82b547df5 service nova] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:09:28.763815 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 696/1371] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:09:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:09:29.075851 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d75ddb-1110-4575-a18d-53d2618a3806 None None] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:09:29.076573 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d75ddb-1110-4575-a18d-53d2618a3806 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:09:29.077416 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d75ddb-1110-4575-a18d-53d2618a3806 None None] PATH_INFO: `/` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:09:29.079253 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 676/1372] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 05:09:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 05:09:29.084809 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0da9061-99a0-4469-94a7-b4a68510611d None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:09:29.085100 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0da9061-99a0-4469-94a7-b4a68510611d None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:09:29.085604 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0da9061-99a0-4469-94a7-b4a68510611d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:09:29.105496 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-a0da9061-99a0-4469-94a7-b4a68510611d None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:09:29.106729 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-a0da9061-99a0-4469-94a7-b4a68510611d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:09:29.115262 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 697/1373] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 05:09:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:09:29.466648 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769d2357-dded-42f7-8f98-b394bc472834 None None] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:09:29.466648 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769d2357-dded-42f7-8f98-b394bc472834 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:09:29.466648 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769d2357-dded-42f7-8f98-b394bc472834 None None] PATH_INFO: `/` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:09:29.467712 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 677/1374] 10.222.0.23 () {54 vars in 963 bytes} [Sat Jun 18 05:09:29 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 05:09:29.473220 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b948a39-8218-445b-9ad3-63c1d4808ae6 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:09:29.473633 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b948a39-8218-445b-9ad3-63c1d4808ae6 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:09:29.474008 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b948a39-8218-445b-9ad3-63c1d4808ae6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:09:29.481910 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:09:29.525453 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-8b948a39-8218-445b-9ad3-63c1d4808ae6 None None] MFA Rules not processed for user `d9de0428abc8400aa338aba4f7c34c94`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:09:29.526786 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-8b948a39-8218-445b-9ad3-63c1d4808ae6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:09:29.590424 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 698/1375] 10.222.0.23 () {58 vars in 1060 bytes} [Sat Jun 18 05:09:29 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:09:29.597866 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769d2357-dded-42f7-8f98-b394bc472834 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:09:29.609166 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769d2357-dded-42f7-8f98-b394bc472834 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:09:29.613699 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7978a3e-55eb-468c-bafd-edaeb879daba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9de0428abc8400aa338aba4f7c34c94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:09:29.614341 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7978a3e-55eb-468c-bafd-edaeb879daba service cinder] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:09:29.614628 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7978a3e-55eb-468c-bafd-edaeb879daba service cinder] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:09:29.614904 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7978a3e-55eb-468c-bafd-edaeb879daba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:09:29.624987 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7978a3e-55eb-468c-bafd-edaeb879daba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:09:29.626471 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7978a3e-55eb-468c-bafd-edaeb879daba service cinder] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:09:29.636755 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 678/1376] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:09:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:09:31.022252 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb5d11a-28c4-405f-a2c8-7bb9f1d6b42c None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:09:31.022803 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb5d11a-28c4-405f-a2c8-7bb9f1d6b42c None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:09:31.022803 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb5d11a-28c4-405f-a2c8-7bb9f1d6b42c None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:09:31.024275 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 699/1377] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 05:09:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 05:09:31.029408 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5716a4e9-c0f1-489d-8d13-7af3707504d9 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:09:31.029815 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5716a4e9-c0f1-489d-8d13-7af3707504d9 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:09:31.029989 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5716a4e9-c0f1-489d-8d13-7af3707504d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:09:31.047819 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-5716a4e9-c0f1-489d-8d13-7af3707504d9 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:09:31.049013 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-5716a4e9-c0f1-489d-8d13-7af3707504d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:09:31.057007 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 679/1378] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 05:09:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:09:31.101811 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb5d11a-28c4-405f-a2c8-7bb9f1d6b42c None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:09:31.117613 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb5d11a-28c4-405f-a2c8-7bb9f1d6b42c None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:09:31.123686 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7f1c81-5e50-4d9c-aba9-bab1b76e6c0b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15e7f45ee0744c1d9602187eac8e8b80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:09:31.124324 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7f1c81-5e50-4d9c-aba9-bab1b76e6c0b service nova] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:09:31.124601 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7f1c81-5e50-4d9c-aba9-bab1b76e6c0b service nova] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:09:31.124871 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7f1c81-5e50-4d9c-aba9-bab1b76e6c0b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:09:31.133009 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7f1c81-5e50-4d9c-aba9-bab1b76e6c0b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:09:31.135010 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7f1c81-5e50-4d9c-aba9-bab1b76e6c0b service nova] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:09:31.144331 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 700/1379] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:09:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:09:33.388877 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57185163-1762-47a6-8c2a-0dff2b2f122a None None] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:09:33.391816 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57185163-1762-47a6-8c2a-0dff2b2f122a None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:09:33.392112 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57185163-1762-47a6-8c2a-0dff2b2f122a None None] PATH_INFO: `/` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:09:33.393625 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 680/1380] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 05:09:33 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 05:09:33.399556 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c0b03a-0801-4a31-8700-ac8b68d44f0f None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:09:33.400367 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c0b03a-0801-4a31-8700-ac8b68d44f0f None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:09:33.400791 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c0b03a-0801-4a31-8700-ac8b68d44f0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:09:33.419550 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-c6c0b03a-0801-4a31-8700-ac8b68d44f0f None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:09:33.420819 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-c6c0b03a-0801-4a31-8700-ac8b68d44f0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:09:33.429229 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 701/1381] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 05:09:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:09:33.473670 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57185163-1762-47a6-8c2a-0dff2b2f122a None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:09:33.491586 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57185163-1762-47a6-8c2a-0dff2b2f122a None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:09:33.498179 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1c25e4-772b-42e2-8855-bf441b8cd73c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15e7f45ee0744c1d9602187eac8e8b80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:09:33.498924 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1c25e4-772b-42e2-8855-bf441b8cd73c service nova] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:09:33.499554 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1c25e4-772b-42e2-8855-bf441b8cd73c service nova] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:09:33.499896 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1c25e4-772b-42e2-8855-bf441b8cd73c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:09:33.509199 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1c25e4-772b-42e2-8855-bf441b8cd73c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:09:33.511203 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1c25e4-772b-42e2-8855-bf441b8cd73c service nova] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:09:33.521783 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 681/1382] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:09:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:09:35.701987 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999a715b-44b1-40d4-8bf7-977c07e66abc None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:09:35.702547 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999a715b-44b1-40d4-8bf7-977c07e66abc None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:09:35.702547 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999a715b-44b1-40d4-8bf7-977c07e66abc None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:09:35.703909 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 702/1383] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 05:09:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 05:09:35.709139 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66340420-34d9-47ae-8a17-ceaf1ae32df3 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:09:35.709639 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66340420-34d9-47ae-8a17-ceaf1ae32df3 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:09:35.709728 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66340420-34d9-47ae-8a17-ceaf1ae32df3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:09:35.728718 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-66340420-34d9-47ae-8a17-ceaf1ae32df3 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:09:35.731770 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-66340420-34d9-47ae-8a17-ceaf1ae32df3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:09:35.738065 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 682/1384] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 05:09:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:09:35.780573 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999a715b-44b1-40d4-8bf7-977c07e66abc None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:09:35.795219 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999a715b-44b1-40d4-8bf7-977c07e66abc None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:09:35.800671 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6be0c3-dae3-4eaa-865e-88e3f851d883 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15e7f45ee0744c1d9602187eac8e8b80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:09:35.801299 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6be0c3-dae3-4eaa-865e-88e3f851d883 service nova] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:09:35.801613 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6be0c3-dae3-4eaa-865e-88e3f851d883 service nova] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:09:35.801917 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6be0c3-dae3-4eaa-865e-88e3f851d883 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:09:35.810828 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6be0c3-dae3-4eaa-865e-88e3f851d883 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:09:35.812462 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6be0c3-dae3-4eaa-865e-88e3f851d883 service nova] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:09:35.823512 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 703/1385] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:09:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:09:37.975095 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20116695-ffbb-46ab-afaa-d7669fcc8cb8 None None] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:09:37.975766 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20116695-ffbb-46ab-afaa-d7669fcc8cb8 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:09:37.975766 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20116695-ffbb-46ab-afaa-d7669fcc8cb8 None None] PATH_INFO: `/` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:09:37.977107 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 683/1386] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 05:09:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 05:09:37.983774 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9046e2ec-912f-4766-b1e7-3511b2fc39ce None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:09:37.984203 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9046e2ec-912f-4766-b1e7-3511b2fc39ce None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:09:37.984545 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9046e2ec-912f-4766-b1e7-3511b2fc39ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:09:38.006853 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-9046e2ec-912f-4766-b1e7-3511b2fc39ce None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:09:38.008345 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-9046e2ec-912f-4766-b1e7-3511b2fc39ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:09:38.017857 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 704/1387] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 05:09:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:09:38.070944 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20116695-ffbb-46ab-afaa-d7669fcc8cb8 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:09:38.085716 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20116695-ffbb-46ab-afaa-d7669fcc8cb8 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:09:38.092197 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a188279e-ebf3-407d-83ff-0b607f7964e5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15e7f45ee0744c1d9602187eac8e8b80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:09:38.092855 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a188279e-ebf3-407d-83ff-0b607f7964e5 service nova] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:09:38.093135 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a188279e-ebf3-407d-83ff-0b607f7964e5 service nova] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:09:38.093440 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a188279e-ebf3-407d-83ff-0b607f7964e5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:09:38.101543 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a188279e-ebf3-407d-83ff-0b607f7964e5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:09:38.104250 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a188279e-ebf3-407d-83ff-0b607f7964e5 service nova] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:09:38.113753 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 684/1388] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:09:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:09:40.632708 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc946d2-8eb3-4d7a-b853-46a9a656940f None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:09:40.633352 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc946d2-8eb3-4d7a-b853-46a9a656940f None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:09:40.633352 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc946d2-8eb3-4d7a-b853-46a9a656940f None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:09:40.635915 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 705/1389] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 05:09:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 05:09:40.641273 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084a37a2-75ef-4755-8300-49cfd87a7f0c None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:09:40.641853 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084a37a2-75ef-4755-8300-49cfd87a7f0c None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:09:40.641964 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084a37a2-75ef-4755-8300-49cfd87a7f0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:09:40.660538 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-084a37a2-75ef-4755-8300-49cfd87a7f0c None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:09:40.662179 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-084a37a2-75ef-4755-8300-49cfd87a7f0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:09:40.671051 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 685/1390] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 05:09:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:09:40.720002 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc946d2-8eb3-4d7a-b853-46a9a656940f None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:09:40.737295 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc946d2-8eb3-4d7a-b853-46a9a656940f None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:09:40.744548 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4df989-2379-4659-8f96-b629861fa5ad service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15e7f45ee0744c1d9602187eac8e8b80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:09:40.745243 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4df989-2379-4659-8f96-b629861fa5ad service nova] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:09:40.745704 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4df989-2379-4659-8f96-b629861fa5ad service nova] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:09:40.746048 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4df989-2379-4659-8f96-b629861fa5ad service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:09:40.756313 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4df989-2379-4659-8f96-b629861fa5ad service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:09:40.758072 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4df989-2379-4659-8f96-b629861fa5ad service nova] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:09:40.768601 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 706/1391] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:09:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:09:43.107266 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6251856-0ab8-49bb-9727-50c223f1976f None None] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:09:43.107795 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6251856-0ab8-49bb-9727-50c223f1976f None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:09:43.107795 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6251856-0ab8-49bb-9727-50c223f1976f None None] PATH_INFO: `/` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:09:43.109207 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 686/1392] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 05:09:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 05:09:43.114712 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c912dd3-5f9a-4a38-bd07-cb2362047359 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:09:43.115131 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c912dd3-5f9a-4a38-bd07-cb2362047359 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:09:43.115273 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c912dd3-5f9a-4a38-bd07-cb2362047359 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:09:43.134627 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-7c912dd3-5f9a-4a38-bd07-cb2362047359 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:09:43.136170 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-7c912dd3-5f9a-4a38-bd07-cb2362047359 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:09:43.144221 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 707/1393] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 05:09:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:09:43.188639 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6251856-0ab8-49bb-9727-50c223f1976f None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:09:43.203344 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6251856-0ab8-49bb-9727-50c223f1976f None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:09:43.208843 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c179c047-3e05-48fc-9ea3-2877a9136b71 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15e7f45ee0744c1d9602187eac8e8b80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:09:43.209689 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c179c047-3e05-48fc-9ea3-2877a9136b71 service nova] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:09:43.209981 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c179c047-3e05-48fc-9ea3-2877a9136b71 service nova] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:09:43.210262 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c179c047-3e05-48fc-9ea3-2877a9136b71 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:09:43.219654 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c179c047-3e05-48fc-9ea3-2877a9136b71 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:09:43.221242 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c179c047-3e05-48fc-9ea3-2877a9136b71 service nova] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:09:43.231362 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 687/1394] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:09:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:09:45.126476 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466434ff-f9c8-4d34-b3e1-b6e986b2e0d1 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:09:45.127031 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466434ff-f9c8-4d34-b3e1-b6e986b2e0d1 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:09:45.127031 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466434ff-f9c8-4d34-b3e1-b6e986b2e0d1 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:09:45.128454 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 708/1395] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 05:09:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 05:09:45.134680 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebb2daf-0c61-4a67-b449-551378893f92 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:09:45.135492 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebb2daf-0c61-4a67-b449-551378893f92 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:09:45.135841 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebb2daf-0c61-4a67-b449-551378893f92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:09:45.161169 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-0ebb2daf-0c61-4a67-b449-551378893f92 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:09:45.162379 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-0ebb2daf-0c61-4a67-b449-551378893f92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:09:45.172676 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 688/1396] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 05:09:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:09:45.237679 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74087e9f-d90f-4216-98ab-c7d1cb01ad98 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:09:45.237966 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74087e9f-d90f-4216-98ab-c7d1cb01ad98 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:09:45.238251 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74087e9f-d90f-4216-98ab-c7d1cb01ad98 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:09:45.239664 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 709/1397] 10.222.0.23 () {54 vars in 962 bytes} [Sat Jun 18 05:09:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 05:09:45.244940 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46115554-f218-4a8d-b7f1-c66d955ce36d None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:09:45.245232 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46115554-f218-4a8d-b7f1-c66d955ce36d None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:09:45.245558 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46115554-f218-4a8d-b7f1-c66d955ce36d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:09:45.298848 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-46115554-f218-4a8d-b7f1-c66d955ce36d None None] MFA Rules not processed for user `4638e5d1fa124667b07054a87c2d353e`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:09:45.300054 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-46115554-f218-4a8d-b7f1-c66d955ce36d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:09:45.364065 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 689/1398] 10.222.0.23 () {58 vars in 1059 bytes} [Sat Jun 18 05:09:45 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:09:45.373455 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74087e9f-d90f-4216-98ab-c7d1cb01ad98 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:09:45.385216 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74087e9f-d90f-4216-98ab-c7d1cb01ad98 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:09:45.389920 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759bad74-7411-4910-8a93-54377de6761c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4638e5d1fa124667b07054a87c2d353e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:09:45.390570 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759bad74-7411-4910-8a93-54377de6761c service glance] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:09:45.391050 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759bad74-7411-4910-8a93-54377de6761c service glance] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:09:45.391262 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759bad74-7411-4910-8a93-54377de6761c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:09:45.401785 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759bad74-7411-4910-8a93-54377de6761c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:09:45.403352 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759bad74-7411-4910-8a93-54377de6761c service glance] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:09:45.414085 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 710/1399] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:09:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 18 05:09:45.575450 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4b4e4f-48c4-419b-9ef7-47cd57b27ff5 None None] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:09:45.575743 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4b4e4f-48c4-419b-9ef7-47cd57b27ff5 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:09:45.576021 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4b4e4f-48c4-419b-9ef7-47cd57b27ff5 None None] PATH_INFO: `/` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:09:45.577442 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 690/1400] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 05:09:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 05:09:45.582794 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6c8f36-0588-4dce-97a0-b9c611ccc8cb None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:09:45.583082 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6c8f36-0588-4dce-97a0-b9c611ccc8cb None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:09:45.583365 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6c8f36-0588-4dce-97a0-b9c611ccc8cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:09:45.602514 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-5c6c8f36-0588-4dce-97a0-b9c611ccc8cb None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:09:45.603935 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-5c6c8f36-0588-4dce-97a0-b9c611ccc8cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:09:45.612285 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 711/1401] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 05:09:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:09:45.659493 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4b4e4f-48c4-419b-9ef7-47cd57b27ff5 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:09:45.674307 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4b4e4f-48c4-419b-9ef7-47cd57b27ff5 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:09:45.679704 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961cd9f7-06dc-4722-96d2-7b5163e3de40 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15e7f45ee0744c1d9602187eac8e8b80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:09:45.680871 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961cd9f7-06dc-4722-96d2-7b5163e3de40 service nova] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:09:45.681160 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961cd9f7-06dc-4722-96d2-7b5163e3de40 service nova] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:09:45.681461 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961cd9f7-06dc-4722-96d2-7b5163e3de40 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:09:45.689887 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961cd9f7-06dc-4722-96d2-7b5163e3de40 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:09:45.691477 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961cd9f7-06dc-4722-96d2-7b5163e3de40 service nova] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:09:45.702616 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 691/1402] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:09:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:09:46.285990 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9898aa55-e9a2-4d60-859b-282e626e6ef3 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:09:46.286442 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9898aa55-e9a2-4d60-859b-282e626e6ef3 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:09:46.286687 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9898aa55-e9a2-4d60-859b-282e626e6ef3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:09:46.334538 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-9898aa55-e9a2-4d60-859b-282e626e6ef3 None None] MFA Rules not processed for user `e817fafd0c1b4bf9944481cf116f1260`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:09:46.335740 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-9898aa55-e9a2-4d60-859b-282e626e6ef3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:09:46.371334 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:09:46.408270 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 712/1403] 10.222.0.23 () {58 vars in 1009 bytes} [Sat Jun 18 05:09:46 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:09:46.454566 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af9f0cd-b2d3-4d44-8deb-71545277d293 None None] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:09:46.455034 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af9f0cd-b2d3-4d44-8deb-71545277d293 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:09:46.455136 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af9f0cd-b2d3-4d44-8deb-71545277d293 None None] PATH_INFO: `/` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:09:46.456478 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 692/1404] 10.222.0.23 () {54 vars in 942 bytes} [Sat Jun 18 05:09:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 05:09:46.464023 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90b29b9-b3a0-4d32-a553-dd92abca7a11 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:09:46.464314 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90b29b9-b3a0-4d32-a553-dd92abca7a11 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:09:46.464593 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90b29b9-b3a0-4d32-a553-dd92abca7a11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:09:46.514630 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-d90b29b9-b3a0-4d32-a553-dd92abca7a11 None None] MFA Rules not processed for user `16b49e866398400580ae48aa5a96fc04`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:09:46.515827 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-d90b29b9-b3a0-4d32-a553-dd92abca7a11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:09:46.587245 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 713/1405] 10.222.0.23 () {58 vars in 1039 bytes} [Sat Jun 18 05:09:46 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:09:46.596874 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af9f0cd-b2d3-4d44-8deb-71545277d293 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:09:46.604350 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=99 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:09:46.610641 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af9f0cd-b2d3-4d44-8deb-71545277d293 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:09:46.615920 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a7bd6c-0a2c-40cd-b07c-05a97f82e2b2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16b49e866398400580ae48aa5a96fc04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:09:46.621061 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a7bd6c-0a2c-40cd-b07c-05a97f82e2b2 service swift] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:09:46.621061 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a7bd6c-0a2c-40cd-b07c-05a97f82e2b2 service swift] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:09:46.621061 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a7bd6c-0a2c-40cd-b07c-05a97f82e2b2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:09:46.625946 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a7bd6c-0a2c-40cd-b07c-05a97f82e2b2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:09:46.627576 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a7bd6c-0a2c-40cd-b07c-05a97f82e2b2 service swift] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:09:46.636905 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 693/1406] 10.222.0.23 () {60 vars in 1384 bytes} [Sat Jun 18 05:09:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 18 05:09:47.975189 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc94cbe2-f7ef-42f8-9787-b75ffa5c7eca None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:09:47.975731 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc94cbe2-f7ef-42f8-9787-b75ffa5c7eca None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:09:47.975731 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc94cbe2-f7ef-42f8-9787-b75ffa5c7eca None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:09:47.977190 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 714/1407] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 05:09:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 05:09:47.983220 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee95d11d-c4a7-446a-8d67-d9d8dc59cf9e None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:09:47.983655 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee95d11d-c4a7-446a-8d67-d9d8dc59cf9e None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:09:47.983913 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee95d11d-c4a7-446a-8d67-d9d8dc59cf9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:09:48.003605 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-ee95d11d-c4a7-446a-8d67-d9d8dc59cf9e None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:09:48.004833 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-ee95d11d-c4a7-446a-8d67-d9d8dc59cf9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:09:48.017776 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 694/1408] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 05:09:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:09:48.059295 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc94cbe2-f7ef-42f8-9787-b75ffa5c7eca None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:09:48.075858 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc94cbe2-f7ef-42f8-9787-b75ffa5c7eca None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:09:48.082020 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f46f96-bc36-4322-ae72-2458b66cfea9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15e7f45ee0744c1d9602187eac8e8b80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:09:48.082739 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f46f96-bc36-4322-ae72-2458b66cfea9 service nova] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:09:48.083031 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f46f96-bc36-4322-ae72-2458b66cfea9 service nova] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:09:48.083310 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f46f96-bc36-4322-ae72-2458b66cfea9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:09:48.093041 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f46f96-bc36-4322-ae72-2458b66cfea9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:09:48.094946 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f46f96-bc36-4322-ae72-2458b66cfea9 service nova] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:09:48.106285 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 715/1409] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:09:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:09:57.996571 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=8 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:09:58.101118 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=62 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:10:10.082901 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdd636c-66d4-42d8-a23b-2b9f4ff3ac16 None None] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:10:10.083476 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdd636c-66d4-42d8-a23b-2b9f4ff3ac16 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:10:10.083476 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdd636c-66d4-42d8-a23b-2b9f4ff3ac16 None None] PATH_INFO: `/` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:10:10.084842 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 695/1410] 10.222.0.23 () {54 vars in 918 bytes} [Sat Jun 18 05:10:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 05:10:10.089977 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290d00ea-1ed6-427a-898d-73a243db7647 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:10:10.090363 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290d00ea-1ed6-427a-898d-73a243db7647 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:10:10.090534 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290d00ea-1ed6-427a-898d-73a243db7647 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:10:10.108120 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-290d00ea-1ed6-427a-898d-73a243db7647 None None] MFA Rules not processed for user `5323ffd91f864680b0c0084eb3f44d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:10:10.109319 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-290d00ea-1ed6-427a-898d-73a243db7647 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:10:10.117806 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 716/1411] 10.222.0.23 () {58 vars in 1015 bytes} [Sat Jun 18 05:10:10 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:10:10.132234 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e3690a-bcab-4ce5-a170-622ed3d8b825 None None] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:10:10.132524 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e3690a-bcab-4ce5-a170-622ed3d8b825 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:10:10.132801 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e3690a-bcab-4ce5-a170-622ed3d8b825 None None] PATH_INFO: `/` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:10:10.134163 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 696/1412] 10.222.0.23 () {54 vars in 918 bytes} [Sat Jun 18 05:10:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 05:10:10.139582 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5f6966-7b6c-4ca4-ba13-14fd7e1070cd None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:10:10.139863 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5f6966-7b6c-4ca4-ba13-14fd7e1070cd None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:10:10.140151 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5f6966-7b6c-4ca4-ba13-14fd7e1070cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:10:10.157616 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-5f5f6966-7b6c-4ca4-ba13-14fd7e1070cd None None] MFA Rules not processed for user `15e7f45ee0744c1d9602187eac8e8b80`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:10:10.158799 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-5f5f6966-7b6c-4ca4-ba13-14fd7e1070cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:10:10.167879 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 717/1413] 10.222.0.23 () {58 vars in 1015 bytes} [Sat Jun 18 05:10:10 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 18 05:10:12.644145 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130854e7-c25c-4a40-8a38-3b224057cc6c None None] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:10:12.644647 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130854e7-c25c-4a40-8a38-3b224057cc6c None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:10:12.644647 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130854e7-c25c-4a40-8a38-3b224057cc6c None None] PATH_INFO: `/` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:10:12.646065 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 697/1414] 127.0.0.1 () {54 vars in 916 bytes} [Sat Jun 18 05:10:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 05:10:12.651295 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f86a13-5653-4427-a43e-bc21508a0afb None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:10:12.651672 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f86a13-5653-4427-a43e-bc21508a0afb None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:10:12.651846 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f86a13-5653-4427-a43e-bc21508a0afb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:10:12.670598 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-20f86a13-5653-4427-a43e-bc21508a0afb None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:10:12.671803 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-20f86a13-5653-4427-a43e-bc21508a0afb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:10:12.679458 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 718/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Sat Jun 18 05:10:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:10:12.723317 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130854e7-c25c-4a40-8a38-3b224057cc6c None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:10:12.737712 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130854e7-c25c-4a40-8a38-3b224057cc6c None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:10:12.743022 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fb5873-ae9c-4930-8762-c77b6e176751 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15e7f45ee0744c1d9602187eac8e8b80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:10:12.743633 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fb5873-ae9c-4930-8762-c77b6e176751 service nova] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:10:12.743912 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fb5873-ae9c-4930-8762-c77b6e176751 service nova] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:10:12.744188 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fb5873-ae9c-4930-8762-c77b6e176751 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:10:12.752827 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fb5873-ae9c-4930-8762-c77b6e176751 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:10:12.754432 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fb5873-ae9c-4930-8762-c77b6e176751 service nova] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:10:12.764960 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 698/1416] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:10:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:10:13.678874 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dbea85-08e9-4901-8d8f-764e15f4dabd None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:10:13.679416 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dbea85-08e9-4901-8d8f-764e15f4dabd None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:10:13.679416 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dbea85-08e9-4901-8d8f-764e15f4dabd None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:10:13.680812 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 719/1417] 127.0.0.1 () {54 vars in 901 bytes} [Sat Jun 18 05:10:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 05:10:13.686767 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45509400-a284-442e-a898-7d307d76f0c6 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:10:13.687169 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45509400-a284-442e-a898-7d307d76f0c6 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:10:13.687315 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45509400-a284-442e-a898-7d307d76f0c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:10:13.705513 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-45509400-a284-442e-a898-7d307d76f0c6 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:10:13.706707 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-45509400-a284-442e-a898-7d307d76f0c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:10:13.714474 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 699/1418] 127.0.0.1 () {58 vars in 998 bytes} [Sat Jun 18 05:10:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:10:13.869196 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dbea85-08e9-4901-8d8f-764e15f4dabd None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:10:13.883604 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dbea85-08e9-4901-8d8f-764e15f4dabd None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:10:13.889230 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb9917d-e9d6-48d2-968a-731f127c0caf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15e7f45ee0744c1d9602187eac8e8b80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:10:13.889902 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb9917d-e9d6-48d2-968a-731f127c0caf service nova] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:10:13.890167 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb9917d-e9d6-48d2-968a-731f127c0caf service nova] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:10:13.890441 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb9917d-e9d6-48d2-968a-731f127c0caf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:10:13.898518 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb9917d-e9d6-48d2-968a-731f127c0caf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:10:13.900099 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb9917d-e9d6-48d2-968a-731f127c0caf service nova] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:10:13.910125 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 720/1419] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:10:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:10:14.789126 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d63f597-0ab4-4e68-b2f2-42a28abde5c3 None None] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:10:14.789719 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d63f597-0ab4-4e68-b2f2-42a28abde5c3 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:10:14.789719 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d63f597-0ab4-4e68-b2f2-42a28abde5c3 None None] PATH_INFO: `/` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:10:14.791073 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 700/1420] 127.0.0.1 () {54 vars in 901 bytes} [Sat Jun 18 05:10:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 05:10:14.796561 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e96b7c-b0ac-4256-baba-e5e3208ab278 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:10:14.797017 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e96b7c-b0ac-4256-baba-e5e3208ab278 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:10:14.797130 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e96b7c-b0ac-4256-baba-e5e3208ab278 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:10:14.815292 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-e4e96b7c-b0ac-4256-baba-e5e3208ab278 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:10:14.816503 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-e4e96b7c-b0ac-4256-baba-e5e3208ab278 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:10:14.824667 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 721/1421] 127.0.0.1 () {58 vars in 998 bytes} [Sat Jun 18 05:10:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 18 05:10:14.975406 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d63f597-0ab4-4e68-b2f2-42a28abde5c3 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:10:14.989520 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d63f597-0ab4-4e68-b2f2-42a28abde5c3 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:10:14.995084 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf5c588-b632-4f8a-b2cb-0d5fd0249965 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15e7f45ee0744c1d9602187eac8e8b80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:10:15.001552 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf5c588-b632-4f8a-b2cb-0d5fd0249965 service nova] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:10:15.001552 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf5c588-b632-4f8a-b2cb-0d5fd0249965 service nova] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:10:15.001552 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf5c588-b632-4f8a-b2cb-0d5fd0249965 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:10:15.004471 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf5c588-b632-4f8a-b2cb-0d5fd0249965 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:10:15.006080 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf5c588-b632-4f8a-b2cb-0d5fd0249965 service nova] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:10:15.016329 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 701/1422] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:10:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:10:15.873565 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1d1052-cdc1-45a8-bbd9-54b29c6acb71 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:10:15.874104 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1d1052-cdc1-45a8-bbd9-54b29c6acb71 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:10:15.874104 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1d1052-cdc1-45a8-bbd9-54b29c6acb71 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:10:15.875525 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 722/1423] 127.0.0.1 () {54 vars in 901 bytes} [Sat Jun 18 05:10:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 05:10:15.880325 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73085ac-e5a0-4092-bb01-bd73d534cbf5 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:10:15.880702 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73085ac-e5a0-4092-bb01-bd73d534cbf5 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:10:15.880881 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73085ac-e5a0-4092-bb01-bd73d534cbf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:10:15.899493 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-d73085ac-e5a0-4092-bb01-bd73d534cbf5 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:10:15.900681 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-d73085ac-e5a0-4092-bb01-bd73d534cbf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:10:15.909224 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 702/1424] 127.0.0.1 () {58 vars in 998 bytes} [Sat Jun 18 05:10:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:10:16.060128 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1d1052-cdc1-45a8-bbd9-54b29c6acb71 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:10:16.075200 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1d1052-cdc1-45a8-bbd9-54b29c6acb71 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:10:16.080662 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c9e58d-a430-4ed6-98f6-1deb49da4019 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15e7f45ee0744c1d9602187eac8e8b80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:10:16.081293 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c9e58d-a430-4ed6-98f6-1deb49da4019 service nova] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:10:16.081607 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c9e58d-a430-4ed6-98f6-1deb49da4019 service nova] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:10:16.081888 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c9e58d-a430-4ed6-98f6-1deb49da4019 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:10:16.089988 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c9e58d-a430-4ed6-98f6-1deb49da4019 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:10:16.091568 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c9e58d-a430-4ed6-98f6-1deb49da4019 service nova] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:10:16.100992 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 723/1425] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:10:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:10:16.955181 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a359c9-95c1-4b08-ae97-7bb75432a3f9 None None] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:10:16.955736 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a359c9-95c1-4b08-ae97-7bb75432a3f9 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:10:16.955736 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a359c9-95c1-4b08-ae97-7bb75432a3f9 None None] PATH_INFO: `/` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:10:16.957071 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 703/1426] 127.0.0.1 () {54 vars in 901 bytes} [Sat Jun 18 05:10:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 05:10:16.962135 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e435b6cd-05b1-4105-bfee-74a4bf0f862b None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:10:16.962509 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e435b6cd-05b1-4105-bfee-74a4bf0f862b None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:10:16.962688 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e435b6cd-05b1-4105-bfee-74a4bf0f862b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:10:16.980302 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-e435b6cd-05b1-4105-bfee-74a4bf0f862b None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:10:16.981599 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-e435b6cd-05b1-4105-bfee-74a4bf0f862b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:10:16.989415 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 724/1427] 127.0.0.1 () {58 vars in 998 bytes} [Sat Jun 18 05:10:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:10:17.139255 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a359c9-95c1-4b08-ae97-7bb75432a3f9 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:10:17.153619 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a359c9-95c1-4b08-ae97-7bb75432a3f9 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:10:17.158984 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b04e7e-ccb3-4aed-8477-d842f7ef174b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15e7f45ee0744c1d9602187eac8e8b80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:10:17.159637 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b04e7e-ccb3-4aed-8477-d842f7ef174b service nova] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:10:17.159920 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b04e7e-ccb3-4aed-8477-d842f7ef174b service nova] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:10:17.160196 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b04e7e-ccb3-4aed-8477-d842f7ef174b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:10:17.168784 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b04e7e-ccb3-4aed-8477-d842f7ef174b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:10:17.170400 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b04e7e-ccb3-4aed-8477-d842f7ef174b service nova] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:10:17.179734 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 704/1428] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:10:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:10:18.036616 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2aeeb8-3bac-4ee6-bc52-a2a713634e19 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:10:18.037181 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2aeeb8-3bac-4ee6-bc52-a2a713634e19 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:10:18.037181 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2aeeb8-3bac-4ee6-bc52-a2a713634e19 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:10:18.038619 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 725/1429] 127.0.0.1 () {54 vars in 901 bytes} [Sat Jun 18 05:10:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 05:10:18.044433 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9766b0e-8f64-45be-be03-57b4273ed561 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:10:18.044871 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9766b0e-8f64-45be-be03-57b4273ed561 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:10:18.044966 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9766b0e-8f64-45be-be03-57b4273ed561 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:10:18.063577 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-c9766b0e-8f64-45be-be03-57b4273ed561 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:10:18.064755 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-c9766b0e-8f64-45be-be03-57b4273ed561 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:10:18.072428 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 705/1430] 127.0.0.1 () {58 vars in 998 bytes} [Sat Jun 18 05:10:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:10:18.226236 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2aeeb8-3bac-4ee6-bc52-a2a713634e19 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:10:18.241169 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2aeeb8-3bac-4ee6-bc52-a2a713634e19 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:10:18.247183 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d99d5d-484d-4124-a045-fbce7e93acc5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15e7f45ee0744c1d9602187eac8e8b80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:10:18.247799 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d99d5d-484d-4124-a045-fbce7e93acc5 service nova] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:10:18.248099 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d99d5d-484d-4124-a045-fbce7e93acc5 service nova] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:10:18.248358 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d99d5d-484d-4124-a045-fbce7e93acc5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:10:18.256430 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d99d5d-484d-4124-a045-fbce7e93acc5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:10:18.258043 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d99d5d-484d-4124-a045-fbce7e93acc5 service nova] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:10:18.268113 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 726/1431] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:10:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:10:19.198008 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a03a14-eaec-4e8e-b5bb-a35920d0a8b4 None None] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:10:19.198616 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a03a14-eaec-4e8e-b5bb-a35920d0a8b4 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:10:19.198616 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a03a14-eaec-4e8e-b5bb-a35920d0a8b4 None None] PATH_INFO: `/` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:10:19.200638 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 706/1432] 127.0.0.1 () {54 vars in 901 bytes} [Sat Jun 18 05:10:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 05:10:19.205569 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cb2335-2f61-4da4-bf6c-53979bf8d56b None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:10:19.206008 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cb2335-2f61-4da4-bf6c-53979bf8d56b None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:10:19.206122 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cb2335-2f61-4da4-bf6c-53979bf8d56b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:10:19.223835 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-94cb2335-2f61-4da4-bf6c-53979bf8d56b None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:10:19.225026 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-94cb2335-2f61-4da4-bf6c-53979bf8d56b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:10:19.232887 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 727/1433] 127.0.0.1 () {58 vars in 998 bytes} [Sat Jun 18 05:10:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:10:19.393261 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a03a14-eaec-4e8e-b5bb-a35920d0a8b4 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:10:19.407382 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a03a14-eaec-4e8e-b5bb-a35920d0a8b4 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:10:19.418632 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c780d10-720a-4e7c-bea2-6242521a1013 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15e7f45ee0744c1d9602187eac8e8b80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:10:19.418632 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c780d10-720a-4e7c-bea2-6242521a1013 service nova] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:10:19.418632 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c780d10-720a-4e7c-bea2-6242521a1013 service nova] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:10:19.418632 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c780d10-720a-4e7c-bea2-6242521a1013 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:10:19.422737 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c780d10-720a-4e7c-bea2-6242521a1013 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:10:19.424350 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c780d10-720a-4e7c-bea2-6242521a1013 service nova] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:10:19.434298 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 707/1434] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:10:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:10:20.274826 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5666dd-34c5-452d-950a-340f6e796363 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:10:20.275405 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5666dd-34c5-452d-950a-340f6e796363 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:10:20.275405 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5666dd-34c5-452d-950a-340f6e796363 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:10:20.276753 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 728/1435] 127.0.0.1 () {54 vars in 901 bytes} [Sat Jun 18 05:10:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 05:10:20.282274 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd67dd1-3882-45c7-a00c-c935ce29a800 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:10:20.282661 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd67dd1-3882-45c7-a00c-c935ce29a800 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:10:20.282828 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd67dd1-3882-45c7-a00c-c935ce29a800 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:10:20.300651 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-fcd67dd1-3882-45c7-a00c-c935ce29a800 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:10:20.301894 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-fcd67dd1-3882-45c7-a00c-c935ce29a800 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:10:20.309652 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 708/1436] 127.0.0.1 () {58 vars in 998 bytes} [Sat Jun 18 05:10:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:10:20.459486 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5666dd-34c5-452d-950a-340f6e796363 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:10:20.474972 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5666dd-34c5-452d-950a-340f6e796363 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:10:20.480308 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3428af91-0623-4c1c-9d92-0fdedffe41c9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15e7f45ee0744c1d9602187eac8e8b80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:10:20.481571 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3428af91-0623-4c1c-9d92-0fdedffe41c9 service nova] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:10:20.481801 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3428af91-0623-4c1c-9d92-0fdedffe41c9 service nova] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:10:20.482077 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3428af91-0623-4c1c-9d92-0fdedffe41c9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:10:20.490284 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3428af91-0623-4c1c-9d92-0fdedffe41c9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:10:20.491861 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3428af91-0623-4c1c-9d92-0fdedffe41c9 service nova] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:10:20.502156 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 729/1437] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:10:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:10:21.354162 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3912a182-255b-4bc7-b5de-a09a8c971ea5 None None] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:10:21.354691 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3912a182-255b-4bc7-b5de-a09a8c971ea5 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:10:21.354691 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3912a182-255b-4bc7-b5de-a09a8c971ea5 None None] PATH_INFO: `/` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:10:21.356484 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 709/1438] 127.0.0.1 () {54 vars in 901 bytes} [Sat Jun 18 05:10:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 05:10:21.361297 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a19c7b2-ef7c-4be6-a920-84f8b2f4ecf4 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:10:21.361706 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a19c7b2-ef7c-4be6-a920-84f8b2f4ecf4 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:10:21.361892 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a19c7b2-ef7c-4be6-a920-84f8b2f4ecf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:10:21.380203 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-2a19c7b2-ef7c-4be6-a920-84f8b2f4ecf4 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:10:21.381455 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-2a19c7b2-ef7c-4be6-a920-84f8b2f4ecf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:10:21.390714 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 730/1439] 127.0.0.1 () {58 vars in 998 bytes} [Sat Jun 18 05:10:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:10:21.540390 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3912a182-255b-4bc7-b5de-a09a8c971ea5 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:10:21.554683 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3912a182-255b-4bc7-b5de-a09a8c971ea5 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:10:21.560666 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3d54aa-9289-4735-8196-cc1be2576484 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15e7f45ee0744c1d9602187eac8e8b80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:10:21.561288 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3d54aa-9289-4735-8196-cc1be2576484 service nova] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:10:21.561601 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3d54aa-9289-4735-8196-cc1be2576484 service nova] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:10:21.561880 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3d54aa-9289-4735-8196-cc1be2576484 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:10:21.569943 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3d54aa-9289-4735-8196-cc1be2576484 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:10:21.571553 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3d54aa-9289-4735-8196-cc1be2576484 service nova] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:10:21.582910 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 710/1440] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:10:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:10:23.526314 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b317742-6a0f-4a84-9a65-ed5e70ee3591 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:10:23.526910 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b317742-6a0f-4a84-9a65-ed5e70ee3591 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:10:23.526910 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b317742-6a0f-4a84-9a65-ed5e70ee3591 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:10:23.529251 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 731/1441] 127.0.0.1 () {54 vars in 916 bytes} [Sat Jun 18 05:10:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 05:10:23.534799 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec01504-3dd0-409a-a954-1a1e12224700 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:10:23.535233 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec01504-3dd0-409a-a954-1a1e12224700 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:10:23.535357 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec01504-3dd0-409a-a954-1a1e12224700 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:10:23.553367 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-0ec01504-3dd0-409a-a954-1a1e12224700 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:10:23.554585 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-0ec01504-3dd0-409a-a954-1a1e12224700 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:10:23.562407 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 711/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Sat Jun 18 05:10:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 18 05:10:23.601645 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b317742-6a0f-4a84-9a65-ed5e70ee3591 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:10:23.603354 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-0b317742-6a0f-4a84-9a65-ed5e70ee3591 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:10:23.616828 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b317742-6a0f-4a84-9a65-ed5e70ee3591 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:10:23.620316 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d186b155-52e5-40f7-a2b9-83c3eebf46dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c4d7cde8f4ad9aa479bb09e325db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:10:23.620937 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d186b155-52e5-40f7-a2b9-83c3eebf46dc service neutron] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:10:23.622190 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d186b155-52e5-40f7-a2b9-83c3eebf46dc service neutron] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:10:23.622470 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d186b155-52e5-40f7-a2b9-83c3eebf46dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:10:23.631198 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d186b155-52e5-40f7-a2b9-83c3eebf46dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:10:23.632644 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d186b155-52e5-40f7-a2b9-83c3eebf46dc service neutron] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:10:23.643316 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 732/1443] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:10:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:10:25.848685 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a269b8d-14cd-48da-a668-242d625fd7e1 None None] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:10:25.849255 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a269b8d-14cd-48da-a668-242d625fd7e1 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:10:25.849255 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a269b8d-14cd-48da-a668-242d625fd7e1 None None] PATH_INFO: `/` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:10:25.850648 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 712/1444] 127.0.0.1 () {54 vars in 916 bytes} [Sat Jun 18 05:10:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 05:10:25.855996 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cd0c50-d09c-49a3-a848-55e66a3fcf12 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:10:25.856539 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cd0c50-d09c-49a3-a848-55e66a3fcf12 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:10:25.856539 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cd0c50-d09c-49a3-a848-55e66a3fcf12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:10:25.874496 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-a3cd0c50-d09c-49a3-a848-55e66a3fcf12 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:10:25.875690 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-a3cd0c50-d09c-49a3-a848-55e66a3fcf12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:10:25.883412 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 733/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Sat Jun 18 05:10:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:10:25.921951 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a269b8d-14cd-48da-a668-242d625fd7e1 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:10:25.932727 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a269b8d-14cd-48da-a668-242d625fd7e1 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:10:25.936897 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b479749b-9933-46e9-842b-00557afc8d11 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c4d7cde8f4ad9aa479bb09e325db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:10:25.937570 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b479749b-9933-46e9-842b-00557afc8d11 service neutron] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:10:25.937859 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b479749b-9933-46e9-842b-00557afc8d11 service neutron] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:10:25.938136 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b479749b-9933-46e9-842b-00557afc8d11 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:10:25.946922 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b479749b-9933-46e9-842b-00557afc8d11 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:10:25.948349 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b479749b-9933-46e9-842b-00557afc8d11 service neutron] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:10:25.957817 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 713/1446] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:10:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:10:28.324188 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ffde23-b08c-4fe8-9ab4-c4a63f6edd1d None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:10:28.324768 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ffde23-b08c-4fe8-9ab4-c4a63f6edd1d None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:10:28.324884 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ffde23-b08c-4fe8-9ab4-c4a63f6edd1d None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:10:28.326239 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 734/1447] 127.0.0.1 () {54 vars in 916 bytes} [Sat Jun 18 05:10:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 05:10:28.331511 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f6c933-4488-445d-9b35-e491c9919735 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:10:28.331934 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f6c933-4488-445d-9b35-e491c9919735 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:10:28.332067 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f6c933-4488-445d-9b35-e491c9919735 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:10:28.350198 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-73f6c933-4488-445d-9b35-e491c9919735 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:10:28.351385 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-73f6c933-4488-445d-9b35-e491c9919735 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:10:28.361262 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 714/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Sat Jun 18 05:10:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:10:28.399902 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ffde23-b08c-4fe8-9ab4-c4a63f6edd1d None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:10:28.411459 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ffde23-b08c-4fe8-9ab4-c4a63f6edd1d None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:10:28.414990 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ecadda-39c4-4a9c-b111-5c0c59daa130 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c4d7cde8f4ad9aa479bb09e325db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:10:28.415616 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ecadda-39c4-4a9c-b111-5c0c59daa130 service neutron] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:10:28.415897 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ecadda-39c4-4a9c-b111-5c0c59daa130 service neutron] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:10:28.416174 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ecadda-39c4-4a9c-b111-5c0c59daa130 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:10:28.425804 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ecadda-39c4-4a9c-b111-5c0c59daa130 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:10:28.427243 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ecadda-39c4-4a9c-b111-5c0c59daa130 service neutron] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:10:28.436867 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 735/1449] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:10:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:10:30.584039 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b0cba5-5445-40c1-9acc-c11fbdaac5d1 None None] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:10:30.584568 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b0cba5-5445-40c1-9acc-c11fbdaac5d1 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:10:30.584568 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b0cba5-5445-40c1-9acc-c11fbdaac5d1 None None] PATH_INFO: `/` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:10:30.586028 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 715/1450] 127.0.0.1 () {54 vars in 916 bytes} [Sat Jun 18 05:10:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 05:10:30.591826 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308a0e22-e673-4d9f-b667-9728b77035e8 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:10:30.592226 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308a0e22-e673-4d9f-b667-9728b77035e8 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:10:30.592385 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308a0e22-e673-4d9f-b667-9728b77035e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:10:30.611459 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-308a0e22-e673-4d9f-b667-9728b77035e8 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:10:30.612659 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-308a0e22-e673-4d9f-b667-9728b77035e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:10:30.620546 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 736/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Sat Jun 18 05:10:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:10:30.659322 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b0cba5-5445-40c1-9acc-c11fbdaac5d1 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:10:30.671178 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b0cba5-5445-40c1-9acc-c11fbdaac5d1 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:10:30.674664 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1d919a-6677-4906-b57b-0e65437d0274 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c4d7cde8f4ad9aa479bb09e325db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:10:30.675281 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1d919a-6677-4906-b57b-0e65437d0274 service neutron] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:10:30.675559 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1d919a-6677-4906-b57b-0e65437d0274 service neutron] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:10:30.675827 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1d919a-6677-4906-b57b-0e65437d0274 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:10:30.685467 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1d919a-6677-4906-b57b-0e65437d0274 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:10:30.686885 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1d919a-6677-4906-b57b-0e65437d0274 service neutron] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:10:30.696092 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 716/1452] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:10:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:10:32.774770 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10159546-0c37-4105-b862-4ff2c334b5db None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:10:32.775301 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10159546-0c37-4105-b862-4ff2c334b5db None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:10:32.775461 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10159546-0c37-4105-b862-4ff2c334b5db None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:10:32.776921 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 737/1453] 127.0.0.1 () {54 vars in 916 bytes} [Sat Jun 18 05:10:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 05:10:32.782264 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e040bf2a-31b5-4bf4-97fb-0f204dbb4d6a None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:10:32.782681 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e040bf2a-31b5-4bf4-97fb-0f204dbb4d6a None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:10:32.782824 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e040bf2a-31b5-4bf4-97fb-0f204dbb4d6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:10:32.800340 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-e040bf2a-31b5-4bf4-97fb-0f204dbb4d6a None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:10:32.801565 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-e040bf2a-31b5-4bf4-97fb-0f204dbb4d6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:10:32.809998 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 717/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Sat Jun 18 05:10:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 18 05:10:32.847590 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10159546-0c37-4105-b862-4ff2c334b5db None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:10:32.858284 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10159546-0c37-4105-b862-4ff2c334b5db None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:10:32.861766 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c80cfc-f567-4433-abd2-eba997325052 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c4d7cde8f4ad9aa479bb09e325db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:10:32.862391 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c80cfc-f567-4433-abd2-eba997325052 service neutron] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:10:32.862704 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c80cfc-f567-4433-abd2-eba997325052 service neutron] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:10:32.862977 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c80cfc-f567-4433-abd2-eba997325052 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:10:32.872338 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c80cfc-f567-4433-abd2-eba997325052 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:10:32.873804 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c80cfc-f567-4433-abd2-eba997325052 service neutron] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:10:32.883096 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 738/1455] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:10:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:10:35.049630 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0513b072-124f-4244-83ec-ff2f494ecb60 None None] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:10:35.050186 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0513b072-124f-4244-83ec-ff2f494ecb60 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:10:35.050186 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0513b072-124f-4244-83ec-ff2f494ecb60 None None] PATH_INFO: `/` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:10:35.051559 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 718/1456] 127.0.0.1 () {54 vars in 916 bytes} [Sat Jun 18 05:10:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 05:10:35.057016 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f325980c-9730-4e55-9df9-4cc08a2260ec None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:10:35.057432 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f325980c-9730-4e55-9df9-4cc08a2260ec None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:10:35.057604 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f325980c-9730-4e55-9df9-4cc08a2260ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:10:35.076118 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-f325980c-9730-4e55-9df9-4cc08a2260ec None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:10:35.077304 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-f325980c-9730-4e55-9df9-4cc08a2260ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:10:35.084991 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 739/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Sat Jun 18 05:10:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:10:35.122924 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0513b072-124f-4244-83ec-ff2f494ecb60 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:10:35.133520 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0513b072-124f-4244-83ec-ff2f494ecb60 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:10:35.137836 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850ccfde-b33e-4ea8-aa7e-e9ddba85be42 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c4d7cde8f4ad9aa479bb09e325db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:10:35.138457 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850ccfde-b33e-4ea8-aa7e-e9ddba85be42 service neutron] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:10:35.138742 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850ccfde-b33e-4ea8-aa7e-e9ddba85be42 service neutron] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:10:35.139019 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850ccfde-b33e-4ea8-aa7e-e9ddba85be42 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:10:35.147708 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850ccfde-b33e-4ea8-aa7e-e9ddba85be42 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:10:35.149127 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850ccfde-b33e-4ea8-aa7e-e9ddba85be42 service neutron] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:10:35.158614 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 719/1458] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:10:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:10:37.222014 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371e0708-fdce-4671-8352-51ebcfafa249 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:10:37.222548 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371e0708-fdce-4671-8352-51ebcfafa249 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:10:37.222548 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371e0708-fdce-4671-8352-51ebcfafa249 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:10:37.223947 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 740/1459] 127.0.0.1 () {54 vars in 916 bytes} [Sat Jun 18 05:10:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 05:10:37.229321 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2c33a4-daf0-4de0-b113-695bca2cc0e8 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:10:37.229755 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2c33a4-daf0-4de0-b113-695bca2cc0e8 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:10:37.229923 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2c33a4-daf0-4de0-b113-695bca2cc0e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:10:37.248197 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-db2c33a4-daf0-4de0-b113-695bca2cc0e8 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:10:37.249412 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-db2c33a4-daf0-4de0-b113-695bca2cc0e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:10:37.257338 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 720/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Sat Jun 18 05:10:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:10:37.296093 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371e0708-fdce-4671-8352-51ebcfafa249 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:10:37.306718 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371e0708-fdce-4671-8352-51ebcfafa249 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:10:37.310781 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f72876-d88d-4289-940b-b6352e0c65aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c4d7cde8f4ad9aa479bb09e325db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:10:37.311403 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f72876-d88d-4289-940b-b6352e0c65aa service neutron] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:10:37.311679 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f72876-d88d-4289-940b-b6352e0c65aa service neutron] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:10:37.311947 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f72876-d88d-4289-940b-b6352e0c65aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:10:37.320863 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f72876-d88d-4289-940b-b6352e0c65aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:10:37.322314 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f72876-d88d-4289-940b-b6352e0c65aa service neutron] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:10:37.332136 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 741/1461] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:10:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:10:39.405757 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7643f45-d61e-442b-874e-9a779a391573 None None] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:10:39.406215 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7643f45-d61e-442b-874e-9a779a391573 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:10:39.406344 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7643f45-d61e-442b-874e-9a779a391573 None None] PATH_INFO: `/` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:10:39.407768 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 721/1462] 127.0.0.1 () {54 vars in 916 bytes} [Sat Jun 18 05:10:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 05:10:39.412984 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d6b703-bc31-460f-9289-309c4677d8df None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:10:39.413453 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d6b703-bc31-460f-9289-309c4677d8df None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:10:39.413551 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d6b703-bc31-460f-9289-309c4677d8df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:10:39.420282 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=79 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:10:39.432135 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-48d6b703-bc31-460f-9289-309c4677d8df None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:10:39.433338 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-48d6b703-bc31-460f-9289-309c4677d8df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:10:39.442432 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 742/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Sat Jun 18 05:10:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:10:39.481016 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7643f45-d61e-442b-874e-9a779a391573 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:10:39.486556 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=73 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:10:39.492246 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7643f45-d61e-442b-874e-9a779a391573 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:10:39.495913 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3c38f7-7cea-4ddc-80b1-45131009d06a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c4d7cde8f4ad9aa479bb09e325db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:10:39.497195 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3c38f7-7cea-4ddc-80b1-45131009d06a service neutron] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:10:39.497499 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3c38f7-7cea-4ddc-80b1-45131009d06a service neutron] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:10:39.497776 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3c38f7-7cea-4ddc-80b1-45131009d06a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:10:39.506722 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3c38f7-7cea-4ddc-80b1-45131009d06a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:10:39.508147 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3c38f7-7cea-4ddc-80b1-45131009d06a service neutron] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:10:39.518713 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 722/1464] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:10:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:10:41.577742 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0427698a-f74f-436a-84cd-fe19a11f7ad4 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:10:41.578289 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0427698a-f74f-436a-84cd-fe19a11f7ad4 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:10:41.578410 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0427698a-f74f-436a-84cd-fe19a11f7ad4 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:10:41.579763 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 743/1465] 127.0.0.1 () {54 vars in 916 bytes} [Sat Jun 18 05:10:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 05:10:41.585058 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529ae7fe-5b23-4622-81ea-88e6a3fccb1d None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:10:41.585495 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529ae7fe-5b23-4622-81ea-88e6a3fccb1d None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:10:41.585666 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529ae7fe-5b23-4622-81ea-88e6a3fccb1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:10:41.603739 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-529ae7fe-5b23-4622-81ea-88e6a3fccb1d None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:10:41.604933 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-529ae7fe-5b23-4622-81ea-88e6a3fccb1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:10:41.613357 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 723/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Sat Jun 18 05:10:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:10:41.648782 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0427698a-f74f-436a-84cd-fe19a11f7ad4 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:10:41.660060 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0427698a-f74f-436a-84cd-fe19a11f7ad4 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:10:41.663541 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cb029e-c7b6-4daa-bd95-c3f9e1ef4acf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c4d7cde8f4ad9aa479bb09e325db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:10:41.664165 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cb029e-c7b6-4daa-bd95-c3f9e1ef4acf service neutron] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:10:41.664473 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cb029e-c7b6-4daa-bd95-c3f9e1ef4acf service neutron] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:10:41.664747 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cb029e-c7b6-4daa-bd95-c3f9e1ef4acf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:10:41.674160 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cb029e-c7b6-4daa-bd95-c3f9e1ef4acf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:10:41.675577 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cb029e-c7b6-4daa-bd95-c3f9e1ef4acf service neutron] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:10:41.684946 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 744/1467] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:10:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:10:43.712725 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0869014-a9a5-4ce0-95fe-9f9824609136 None None] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:10:43.713248 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0869014-a9a5-4ce0-95fe-9f9824609136 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:10:43.713345 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0869014-a9a5-4ce0-95fe-9f9824609136 None None] PATH_INFO: `/` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:10:43.714668 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 724/1468] 127.0.0.1 () {54 vars in 916 bytes} [Sat Jun 18 05:10:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 05:10:43.720282 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243ec698-d38d-4e31-85c8-dd181e329f94 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:10:43.720719 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243ec698-d38d-4e31-85c8-dd181e329f94 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:10:43.720858 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243ec698-d38d-4e31-85c8-dd181e329f94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:10:43.738587 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-243ec698-d38d-4e31-85c8-dd181e329f94 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:10:43.739770 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-243ec698-d38d-4e31-85c8-dd181e329f94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:10:43.748055 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 745/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Sat Jun 18 05:10:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:10:43.785624 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0869014-a9a5-4ce0-95fe-9f9824609136 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:10:43.796282 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0869014-a9a5-4ce0-95fe-9f9824609136 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:10:43.799771 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31caa76-faa2-45ee-93eb-b156bc38f15e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c4d7cde8f4ad9aa479bb09e325db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:10:43.800386 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31caa76-faa2-45ee-93eb-b156bc38f15e service neutron] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:10:43.800670 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31caa76-faa2-45ee-93eb-b156bc38f15e service neutron] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:10:43.800944 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31caa76-faa2-45ee-93eb-b156bc38f15e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:10:43.811182 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31caa76-faa2-45ee-93eb-b156bc38f15e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:10:43.812633 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31caa76-faa2-45ee-93eb-b156bc38f15e service neutron] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:10:43.821870 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 725/1470] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:10:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:10:53.733601 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=11 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:10:53.817732 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=14 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:11:03.621204 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6fba52-2000-4b3a-964f-f44d2287313b None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:11:03.621799 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6fba52-2000-4b3a-964f-f44d2287313b None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:11:03.621799 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6fba52-2000-4b3a-964f-f44d2287313b None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:11:03.623274 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 746/1471] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 05:11:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 05:11:03.628806 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acecbbbb-bc3c-49fd-9829-62f53434007b None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:11:03.629305 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acecbbbb-bc3c-49fd-9829-62f53434007b None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:11:03.629429 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acecbbbb-bc3c-49fd-9829-62f53434007b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:11:03.647476 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-acecbbbb-bc3c-49fd-9829-62f53434007b None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:11:03.648596 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-acecbbbb-bc3c-49fd-9829-62f53434007b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:11:03.656833 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 726/1472] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 05:11:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:11:03.704561 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ae7048-7a7a-4844-939f-6d7f3413ee96 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:11:03.704848 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ae7048-7a7a-4844-939f-6d7f3413ee96 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:11:03.705124 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ae7048-7a7a-4844-939f-6d7f3413ee96 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:11:03.706525 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 747/1473] 10.222.0.23 () {54 vars in 962 bytes} [Sat Jun 18 05:11:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 05:11:03.711306 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142612bc-5654-4150-a2b0-30c9a0509117 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:11:03.711592 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142612bc-5654-4150-a2b0-30c9a0509117 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:11:03.711871 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142612bc-5654-4150-a2b0-30c9a0509117 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:11:03.730247 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-142612bc-5654-4150-a2b0-30c9a0509117 None None] MFA Rules not processed for user `4638e5d1fa124667b07054a87c2d353e`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:11:03.731425 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-142612bc-5654-4150-a2b0-30c9a0509117 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:11:03.738215 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 727/1474] 10.222.0.23 () {58 vars in 1059 bytes} [Sat Jun 18 05:11:03 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:11:03.745112 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ae7048-7a7a-4844-939f-6d7f3413ee96 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:11:03.755597 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ae7048-7a7a-4844-939f-6d7f3413ee96 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:11:03.759144 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373eeebf-8da4-4d4d-a7c2-27a255b331f1 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4638e5d1fa124667b07054a87c2d353e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:11:03.759756 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373eeebf-8da4-4d4d-a7c2-27a255b331f1 service glance] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:11:03.760033 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373eeebf-8da4-4d4d-a7c2-27a255b331f1 service glance] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:11:03.760301 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373eeebf-8da4-4d4d-a7c2-27a255b331f1 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:11:03.770112 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373eeebf-8da4-4d4d-a7c2-27a255b331f1 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:11:03.771522 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373eeebf-8da4-4d4d-a7c2-27a255b331f1 service glance] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:11:03.781839 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 748/1475] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:11:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:11:05.924444 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5b8317-da72-42aa-91d0-72d79d4a42ca None None] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:11:05.925003 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5b8317-da72-42aa-91d0-72d79d4a42ca None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:11:05.925003 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5b8317-da72-42aa-91d0-72d79d4a42ca None None] PATH_INFO: `/` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:11:05.926387 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 728/1476] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 05:11:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 05:11:05.931625 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcfb522-c0c2-435e-87de-bba975d7f11d None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:11:05.932023 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcfb522-c0c2-435e-87de-bba975d7f11d None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:11:05.932179 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcfb522-c0c2-435e-87de-bba975d7f11d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:11:05.950275 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-9fcfb522-c0c2-435e-87de-bba975d7f11d None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:11:05.951494 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-9fcfb522-c0c2-435e-87de-bba975d7f11d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:11:05.959276 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 749/1477] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 05:11:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:11:05.982710 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5b8317-da72-42aa-91d0-72d79d4a42ca None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:11:05.995701 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5b8317-da72-42aa-91d0-72d79d4a42ca None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:11:06.000865 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e704a8-0dc1-4824-8823-4669742d8f4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:11:06.001527 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e704a8-0dc1-4824-8823-4669742d8f4e admin admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:11:06.001810 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e704a8-0dc1-4824-8823-4669742d8f4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:11:06.002091 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e704a8-0dc1-4824-8823-4669742d8f4e admin admin] PATH_INFO: `/v3/projects` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:11:06.002820 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e704a8-0dc1-4824-8823-4669742d8f4e admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:11:06.004575 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: WARNING py.warnings [None req-21e704a8-0dc1-4824-8823-4669742d8f4e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 18 05:11:06.004575 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: warnings.warn(msg) Jun 18 05:11:06.004575 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: Jun 18 05:11:06.005768 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e704a8-0dc1-4824-8823-4669742d8f4e admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:11:06.027074 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 729/1478] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Jun 18 05:11:05 2022] GET /identity/v3/projects => generated 2553 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:11:08.089667 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70184de-eff1-4f49-be1e-549c63a88f2c None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:11:08.090206 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70184de-eff1-4f49-be1e-549c63a88f2c None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:11:08.090206 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70184de-eff1-4f49-be1e-549c63a88f2c None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:11:08.092532 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 750/1479] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 05:11:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 05:11:08.097623 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa939825-cfe7-4dda-a08c-d27cbdad3aa2 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:11:08.098020 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa939825-cfe7-4dda-a08c-d27cbdad3aa2 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:11:08.098175 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa939825-cfe7-4dda-a08c-d27cbdad3aa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:11:08.115815 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-fa939825-cfe7-4dda-a08c-d27cbdad3aa2 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:11:08.117002 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-fa939825-cfe7-4dda-a08c-d27cbdad3aa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:11:08.124685 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 730/1480] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 05:11:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:11:08.166920 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70184de-eff1-4f49-be1e-549c63a88f2c None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:11:08.181032 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70184de-eff1-4f49-be1e-549c63a88f2c None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:11:08.188503 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43232278-e658-4d26-acfd-f2e4d82a249c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15e7f45ee0744c1d9602187eac8e8b80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:11:08.188503 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43232278-e658-4d26-acfd-f2e4d82a249c service nova] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:11:08.188503 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43232278-e658-4d26-acfd-f2e4d82a249c service nova] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:11:08.188503 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43232278-e658-4d26-acfd-f2e4d82a249c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:11:08.195646 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43232278-e658-4d26-acfd-f2e4d82a249c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:11:08.197230 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43232278-e658-4d26-acfd-f2e4d82a249c service nova] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:11:08.207561 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 751/1481] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:11:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:11:10.437525 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef4bd1a-f57a-4f6f-9135-b3e0a36aadea None None] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:11:10.438058 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef4bd1a-f57a-4f6f-9135-b3e0a36aadea None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:11:10.438058 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef4bd1a-f57a-4f6f-9135-b3e0a36aadea None None] PATH_INFO: `/` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:11:10.439428 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 731/1482] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 05:11:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 05:11:10.444605 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f3b535-fb4b-4865-9deb-38ec2e92dcd0 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:11:10.445009 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f3b535-fb4b-4865-9deb-38ec2e92dcd0 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:11:10.445161 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f3b535-fb4b-4865-9deb-38ec2e92dcd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:11:10.463492 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-05f3b535-fb4b-4865-9deb-38ec2e92dcd0 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:11:10.464693 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-05f3b535-fb4b-4865-9deb-38ec2e92dcd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:11:10.473405 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 752/1483] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 05:11:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:11:10.510306 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef4bd1a-f57a-4f6f-9135-b3e0a36aadea None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:11:10.522398 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef4bd1a-f57a-4f6f-9135-b3e0a36aadea None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:11:10.526263 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30b4aeb-e5fd-442f-9bb4-7f9727cdd7a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c4d7cde8f4ad9aa479bb09e325db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:11:10.526907 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30b4aeb-e5fd-442f-9bb4-7f9727cdd7a8 service neutron] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:11:10.527192 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30b4aeb-e5fd-442f-9bb4-7f9727cdd7a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:11:10.527469 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30b4aeb-e5fd-442f-9bb4-7f9727cdd7a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:11:10.538279 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30b4aeb-e5fd-442f-9bb4-7f9727cdd7a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:11:10.539722 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30b4aeb-e5fd-442f-9bb4-7f9727cdd7a8 service neutron] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:11:10.549771 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 732/1484] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:11:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:11:12.652935 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd633002-d4ff-421c-9f36-bef081f62cee None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:11:12.653497 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd633002-d4ff-421c-9f36-bef081f62cee None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:11:12.653497 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd633002-d4ff-421c-9f36-bef081f62cee None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:11:12.654982 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 753/1485] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 05:11:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 18 05:11:12.660204 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59580ab2-ac46-409f-a4bb-31aa1102e5d2 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:11:12.660669 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59580ab2-ac46-409f-a4bb-31aa1102e5d2 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:11:12.660774 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59580ab2-ac46-409f-a4bb-31aa1102e5d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:11:12.679523 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-59580ab2-ac46-409f-a4bb-31aa1102e5d2 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:11:12.680709 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-59580ab2-ac46-409f-a4bb-31aa1102e5d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:11:12.689548 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 733/1486] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 05:11:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:11:12.726783 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd633002-d4ff-421c-9f36-bef081f62cee None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:11:12.738800 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd633002-d4ff-421c-9f36-bef081f62cee None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:11:12.742317 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8f70fa-4994-4786-8c8f-462757878904 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c4d7cde8f4ad9aa479bb09e325db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:11:12.742937 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8f70fa-4994-4786-8c8f-462757878904 service neutron] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:11:12.743211 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8f70fa-4994-4786-8c8f-462757878904 service neutron] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:11:12.743483 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8f70fa-4994-4786-8c8f-462757878904 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:11:12.752978 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8f70fa-4994-4786-8c8f-462757878904 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:11:12.754438 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8f70fa-4994-4786-8c8f-462757878904 service neutron] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:11:12.764107 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 754/1487] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:11:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:11:14.980525 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b52de8-141d-41ed-9390-2c552cccd26f None None] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:11:14.981057 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b52de8-141d-41ed-9390-2c552cccd26f None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:11:14.981057 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b52de8-141d-41ed-9390-2c552cccd26f None None] PATH_INFO: `/` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:11:14.983020 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 734/1488] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 05:11:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 05:11:14.988475 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bf3531-3a6c-4a9c-98e4-2711db67f4ae None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:11:14.988886 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bf3531-3a6c-4a9c-98e4-2711db67f4ae None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:11:14.989033 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bf3531-3a6c-4a9c-98e4-2711db67f4ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:11:15.007465 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-03bf3531-3a6c-4a9c-98e4-2711db67f4ae None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:11:15.008663 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-03bf3531-3a6c-4a9c-98e4-2711db67f4ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:11:15.016848 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 755/1489] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 05:11:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:11:15.056757 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b52de8-141d-41ed-9390-2c552cccd26f None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:11:15.067934 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b52de8-141d-41ed-9390-2c552cccd26f None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:11:15.071464 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df0d3d7-21c7-4193-bead-6c7a27e02c9e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c4d7cde8f4ad9aa479bb09e325db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:11:15.072087 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df0d3d7-21c7-4193-bead-6c7a27e02c9e service neutron] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:11:15.072370 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df0d3d7-21c7-4193-bead-6c7a27e02c9e service neutron] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:11:15.072645 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df0d3d7-21c7-4193-bead-6c7a27e02c9e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:11:15.081509 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df0d3d7-21c7-4193-bead-6c7a27e02c9e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:11:15.082928 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df0d3d7-21c7-4193-bead-6c7a27e02c9e service neutron] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:11:15.092293 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 735/1490] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:11:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:11:17.566894 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e074ce76-892d-4941-bde3-b8fc6b44f6cb None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:11:17.567416 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e074ce76-892d-4941-bde3-b8fc6b44f6cb None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:11:17.567512 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e074ce76-892d-4941-bde3-b8fc6b44f6cb None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:11:17.568822 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 756/1491] 10.222.0.23 () {54 vars in 926 bytes} [Sat Jun 18 05:11:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 05:11:17.573985 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82258700-3963-4857-9bf8-3a637a6fd142 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:11:17.574407 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82258700-3963-4857-9bf8-3a637a6fd142 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:11:17.574549 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82258700-3963-4857-9bf8-3a637a6fd142 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:11:17.593202 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-82258700-3963-4857-9bf8-3a637a6fd142 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:11:17.594447 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-82258700-3963-4857-9bf8-3a637a6fd142 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:11:17.602291 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 736/1492] 10.222.0.23 () {58 vars in 1023 bytes} [Sat Jun 18 05:11:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 18 05:11:17.642504 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e074ce76-892d-4941-bde3-b8fc6b44f6cb None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:11:17.652933 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e074ce76-892d-4941-bde3-b8fc6b44f6cb None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:11:17.656858 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce2c834-745c-476d-abbf-238c9965801e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c4d7cde8f4ad9aa479bb09e325db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:11:17.657527 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce2c834-745c-476d-abbf-238c9965801e service neutron] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:11:17.657813 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce2c834-745c-476d-abbf-238c9965801e service neutron] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:11:17.658089 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce2c834-745c-476d-abbf-238c9965801e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:11:17.667023 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce2c834-745c-476d-abbf-238c9965801e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:11:17.668458 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce2c834-745c-476d-abbf-238c9965801e service neutron] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:11:17.677975 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 757/1493] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:11:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:11:27.587871 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=33 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:11:27.673858 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=33 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:12:50.824328 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c84604-691e-4fa6-971a-589956dea477 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:12:50.824910 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c84604-691e-4fa6-971a-589956dea477 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:12:50.824910 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c84604-691e-4fa6-971a-589956dea477 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:12:50.845256 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-c6c84604-691e-4fa6-971a-589956dea477 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:12:50.846578 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-c6c84604-691e-4fa6-971a-589956dea477 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:12:50.854745 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 737/1494] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:12:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:12:50.879796 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4273247e-e2c1-475d-a594-59d70f38d6e2 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:12:50.880301 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4273247e-e2c1-475d-a594-59d70f38d6e2 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:12:50.880440 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4273247e-e2c1-475d-a594-59d70f38d6e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:12:50.899117 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-4273247e-e2c1-475d-a594-59d70f38d6e2 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:12:50.900306 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-4273247e-e2c1-475d-a594-59d70f38d6e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:12:50.909560 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 758/1495] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:12:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:12:50.914075 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c84604-691e-4fa6-971a-589956dea477 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:12:50.923871 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c84604-691e-4fa6-971a-589956dea477 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:12:50.928378 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07f8853-4ed5-4500-bdf8-339bb425702f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:12:50.929020 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07f8853-4ed5-4500-bdf8-339bb425702f admin admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:12:50.929310 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07f8853-4ed5-4500-bdf8-339bb425702f admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:12:50.929619 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07f8853-4ed5-4500-bdf8-339bb425702f admin admin] PATH_INFO: `/v3/domains` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:12:50.930334 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07f8853-4ed5-4500-bdf8-339bb425702f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 05:12:50.930641 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07f8853-4ed5-4500-bdf8-339bb425702f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:12:50.931653 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: WARNING py.warnings [None req-a07f8853-4ed5-4500-bdf8-339bb425702f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 18 05:12:50.931653 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: warnings.warn(msg) Jun 18 05:12:50.931653 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: Jun 18 05:12:50.932533 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07f8853-4ed5-4500-bdf8-339bb425702f admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:12:50.944230 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 738/1496] 10.222.0.23 () {58 vars in 1152 bytes} [Sat Jun 18 05:12:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 05:12:50.953028 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4273247e-e2c1-475d-a594-59d70f38d6e2 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:12:50.962299 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4273247e-e2c1-475d-a594-59d70f38d6e2 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:12:50.966816 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e1b666-7955-4574-9ee5-c337ea7f6d5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:12:50.967462 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e1b666-7955-4574-9ee5-c337ea7f6d5d admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:12:50.968402 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e1b666-7955-4574-9ee5-c337ea7f6d5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:12:50.968687 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e1b666-7955-4574-9ee5-c337ea7f6d5d admin admin] PATH_INFO: `/v3/projects` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:12:50.969376 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e1b666-7955-4574-9ee5-c337ea7f6d5d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:12:50.971180 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: WARNING py.warnings [None req-53e1b666-7955-4574-9ee5-c337ea7f6d5d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 18 05:12:50.971180 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: warnings.warn(msg) Jun 18 05:12:50.971180 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: Jun 18 05:12:50.972381 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e1b666-7955-4574-9ee5-c337ea7f6d5d admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:12:51.091399 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 759/1497] 10.222.0.23 () {60 vars in 1152 bytes} [Sat Jun 18 05:12:50 2022] POST /identity/v3/projects => generated 377 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 18 05:12:51.096361 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07f8853-4ed5-4500-bdf8-339bb425702f admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:12:51.106399 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07f8853-4ed5-4500-bdf8-339bb425702f admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:12:51.163482 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6139195a-52fd-46bf-994a-fe7267d4f4ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:12:51.164143 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6139195a-52fd-46bf-994a-fe7267d4f4ca admin admin] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:12:51.164425 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6139195a-52fd-46bf-994a-fe7267d4f4ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:12:51.164703 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6139195a-52fd-46bf-994a-fe7267d4f4ca admin admin] PATH_INFO: `/v3/users` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:12:51.165415 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6139195a-52fd-46bf-994a-fe7267d4f4ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:12:51.166423 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: WARNING py.warnings [None req-6139195a-52fd-46bf-994a-fe7267d4f4ca admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 18 05:12:51.166423 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: warnings.warn(msg) Jun 18 05:12:51.166423 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: Jun 18 05:12:51.167667 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6139195a-52fd-46bf-994a-fe7267d4f4ca admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:12:51.272520 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 739/1498] 10.222.0.23 () {60 vars in 1143 bytes} [Sat Jun 18 05:12:51 2022] POST /identity/v3/users => generated 289 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:12:51.276708 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e1b666-7955-4574-9ee5-c337ea7f6d5d admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:12:51.288212 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e1b666-7955-4574-9ee5-c337ea7f6d5d admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:12:51.292953 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9feb6e-a780-4fea-aa32-a5bf4e385fac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:12:51.293625 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9feb6e-a780-4fea-aa32-a5bf4e385fac admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:12:51.293907 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9feb6e-a780-4fea-aa32-a5bf4e385fac admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:12:51.294180 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9feb6e-a780-4fea-aa32-a5bf4e385fac admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:12:51.294896 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9feb6e-a780-4fea-aa32-a5bf4e385fac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:12:51.295961 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: WARNING py.warnings [None req-ed9feb6e-a780-4fea-aa32-a5bf4e385fac admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 18 05:12:51.295961 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: warnings.warn(msg) Jun 18 05:12:51.295961 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: Jun 18 05:12:51.301169 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9feb6e-a780-4fea-aa32-a5bf4e385fac admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:12:51.314757 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 760/1499] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:12:51 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:12:51.318704 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6139195a-52fd-46bf-994a-fe7267d4f4ca admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:12:51.329270 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6139195a-52fd-46bf-994a-fe7267d4f4ca admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:12:51.333853 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e83cf3-1d6a-4dde-a070-f0e2369294ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:12:51.334501 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e83cf3-1d6a-4dde-a070-f0e2369294ed admin admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:12:51.334785 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e83cf3-1d6a-4dde-a070-f0e2369294ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:12:51.335058 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e83cf3-1d6a-4dde-a070-f0e2369294ed admin admin] PATH_INFO: `/v3/roles` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:12:51.335778 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e83cf3-1d6a-4dde-a070-f0e2369294ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:12:51.336777 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: WARNING py.warnings [None req-32e83cf3-1d6a-4dde-a070-f0e2369294ed admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 18 05:12:51.336777 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: warnings.warn(msg) Jun 18 05:12:51.336777 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: Jun 18 05:12:51.337709 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e83cf3-1d6a-4dde-a070-f0e2369294ed admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:12:51.351044 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 740/1500] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:12:51 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:12:51.355135 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9feb6e-a780-4fea-aa32-a5bf4e385fac admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:12:51.365111 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9feb6e-a780-4fea-aa32-a5bf4e385fac admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:12:51.369662 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c3ce60-aa83-4d0d-aa97-4308363aefc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:12:51.370531 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c3ce60-aa83-4d0d-aa97-4308363aefc6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:12:51.370815 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c3ce60-aa83-4d0d-aa97-4308363aefc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:12:51.371090 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c3ce60-aa83-4d0d-aa97-4308363aefc6 admin admin] PATH_INFO: `/v3/projects/7eef79de2b9745da908815751a816904/users/1469c831bead4925bf3eb5c88b5d84d8/roles/cbf5c4c26898413d93e73b46b41046a3` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:12:51.388340 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c3ce60-aa83-4d0d-aa97-4308363aefc6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7eef79de2b9745da908815751a816904, user_id=1469c831bead4925bf3eb5c88b5d84d8, role_id=cbf5c4c26898413d93e73b46b41046a3)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:12:51.389377 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: WARNING py.warnings [None req-73c3ce60-aa83-4d0d-aa97-4308363aefc6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 18 05:12:51.389377 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: warnings.warn(msg) Jun 18 05:12:51.389377 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: Jun 18 05:12:51.392145 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c3ce60-aa83-4d0d-aa97-4308363aefc6 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:12:51.465650 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 761/1501] 10.222.0.23 () {60 vars in 1482 bytes} [Sat Jun 18 05:12:51 2022] PUT /identity/v3/projects/7eef79de2b9745da908815751a816904/users/1469c831bead4925bf3eb5c88b5d84d8/roles/cbf5c4c26898413d93e73b46b41046a3 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:12:51.477165 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a9fc85-94e6-499f-a1dd-ee02cb228d09 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:12:51.477490 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a9fc85-94e6-499f-a1dd-ee02cb228d09 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:12:51.477775 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a9fc85-94e6-499f-a1dd-ee02cb228d09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:12:51.505281 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-e2a9fc85-94e6-499f-a1dd-ee02cb228d09 None None] MFA Rules not processed for user `1469c831bead4925bf3eb5c88b5d84d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:12:51.506520 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-e2a9fc85-94e6-499f-a1dd-ee02cb228d09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:12:51.569909 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 741/1502] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:12:51 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:12:51.583963 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0098a4f9-b1d7-4390-b884-1efcb578bbc4 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:12:51.584249 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0098a4f9-b1d7-4390-b884-1efcb578bbc4 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:12:51.584525 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0098a4f9-b1d7-4390-b884-1efcb578bbc4 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:12:51.585999 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 762/1503] 10.222.0.23 () {54 vars in 963 bytes} [Sat Jun 18 05:12:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 05:12:51.591020 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940acbcf-5e47-4bf9-87e5-d2f045875169 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:12:51.591306 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940acbcf-5e47-4bf9-87e5-d2f045875169 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:12:51.591584 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940acbcf-5e47-4bf9-87e5-d2f045875169 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:12:51.610200 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-940acbcf-5e47-4bf9-87e5-d2f045875169 None None] MFA Rules not processed for user `d9de0428abc8400aa338aba4f7c34c94`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:12:51.611387 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-940acbcf-5e47-4bf9-87e5-d2f045875169 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:12:51.656217 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:12:51.663639 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 742/1504] 10.222.0.23 () {58 vars in 1060 bytes} [Sat Jun 18 05:12:51 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:12:51.672895 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0098a4f9-b1d7-4390-b884-1efcb578bbc4 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:12:51.683672 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0098a4f9-b1d7-4390-b884-1efcb578bbc4 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:12:51.690798 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf5f55a-e3d4-40c9-ba88-d60143dbc809 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9de0428abc8400aa338aba4f7c34c94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:12:51.690798 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf5f55a-e3d4-40c9-ba88-d60143dbc809 service cinder] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:12:51.690798 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf5f55a-e3d4-40c9-ba88-d60143dbc809 service cinder] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:12:51.691109 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf5f55a-e3d4-40c9-ba88-d60143dbc809 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:12:51.699397 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf5f55a-e3d4-40c9-ba88-d60143dbc809 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:12:51.700865 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf5f55a-e3d4-40c9-ba88-d60143dbc809 service cinder] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:12:51.710068 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 763/1505] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:12:51 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:12:51.753060 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3427a4d-334c-4208-83a0-641b6effb728 None None] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:12:51.753357 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3427a4d-334c-4208-83a0-641b6effb728 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:12:51.753842 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3427a4d-334c-4208-83a0-641b6effb728 None None] PATH_INFO: `/` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:12:51.754956 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 743/1506] 10.222.0.23 () {52 vars in 829 bytes} [Sat Jun 18 05:12:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 05:12:51.758651 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf5f55a-e3d4-40c9-ba88-d60143dbc809 service cinder] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:12:51.821777 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf5f55a-e3d4-40c9-ba88-d60143dbc809 service cinder] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:12:51.826360 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f37ed7e-284c-40b4-96ce-210606b00e7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:12:51.827215 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f37ed7e-284c-40b4-96ce-210606b00e7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:12:51.827494 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f37ed7e-284c-40b4-96ce-210606b00e7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:12:51.827764 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f37ed7e-284c-40b4-96ce-210606b00e7f admin admin] PATH_INFO: `/v3/users/1469c831bead4925bf3eb5c88b5d84d8` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:12:51.828928 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f37ed7e-284c-40b4-96ce-210606b00e7f admin admin] RBAC: Authorizing `identity:delete_user(user_id=1469c831bead4925bf3eb5c88b5d84d8)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:12:51.830030 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: WARNING py.warnings [None req-2f37ed7e-284c-40b4-96ce-210606b00e7f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 18 05:12:51.830030 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: warnings.warn(msg) Jun 18 05:12:51.830030 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: Jun 18 05:12:51.831235 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f37ed7e-284c-40b4-96ce-210606b00e7f admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:12:52.063635 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-2f37ed7e-284c-40b4-96ce-210606b00e7f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1469c831bead4925bf3eb5c88b5d84d8'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:12:52.081917 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-2f37ed7e-284c-40b4-96ce-210606b00e7f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1469c831bead4925bf3eb5c88b5d84d8'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:12:52.082631 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-2f37ed7e-284c-40b4-96ce-210606b00e7f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1469c831bead4925bf3eb5c88b5d84d8'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:12:52.141772 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-2f37ed7e-284c-40b4-96ce-210606b00e7f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1469c831bead4925bf3eb5c88b5d84d8'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:12:52.150530 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:12:52.154751 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-2f37ed7e-284c-40b4-96ce-210606b00e7f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1469c831bead4925bf3eb5c88b5d84d8'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:12:52.161291 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 764/1507] 10.222.0.23 () {58 vars in 1223 bytes} [Sat Jun 18 05:12:51 2022] DELETE /identity/v3/users/1469c831bead4925bf3eb5c88b5d84d8 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:12:52.166011 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df4c237-1231-4d3f-abb4-dd1ba9fea951 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:12:52.169692 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df4c237-1231-4d3f-abb4-dd1ba9fea951 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:12:52.169692 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df4c237-1231-4d3f-abb4-dd1ba9fea951 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:12:52.184678 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-2df4c237-1231-4d3f-abb4-dd1ba9fea951 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:12:52.186023 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-2df4c237-1231-4d3f-abb4-dd1ba9fea951 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:12:52.244865 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 744/1508] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:12:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:12:52.254421 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f37ed7e-284c-40b4-96ce-210606b00e7f admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:12:52.255814 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-2f37ed7e-284c-40b4-96ce-210606b00e7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:12:52.311832 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f37ed7e-284c-40b4-96ce-210606b00e7f admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:12:52.315623 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f282170-ebc6-41db-b617-a44930e747f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c4d7cde8f4ad9aa479bb09e325db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:12:52.316202 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f282170-ebc6-41db-b617-a44930e747f2 service neutron] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:12:52.316485 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f282170-ebc6-41db-b617-a44930e747f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:12:52.316760 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f282170-ebc6-41db-b617-a44930e747f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:12:52.325597 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f282170-ebc6-41db-b617-a44930e747f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:12:52.327038 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f282170-ebc6-41db-b617-a44930e747f2 service neutron] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:12:52.336962 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 765/1509] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:12:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:12:52.800907 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df4c237-1231-4d3f-abb4-dd1ba9fea951 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:12:52.802368 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-2df4c237-1231-4d3f-abb4-dd1ba9fea951 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:12:52.814649 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df4c237-1231-4d3f-abb4-dd1ba9fea951 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:12:52.825596 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f389025-ead3-444b-8ac5-5012fe3ef5c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:12:52.825596 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f389025-ead3-444b-8ac5-5012fe3ef5c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:12:52.825596 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f389025-ead3-444b-8ac5-5012fe3ef5c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:12:52.825596 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f389025-ead3-444b-8ac5-5012fe3ef5c1 admin admin] PATH_INFO: `/v3/projects/7eef79de2b9745da908815751a816904` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:12:52.825946 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f389025-ead3-444b-8ac5-5012fe3ef5c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7eef79de2b9745da908815751a816904)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:12:52.825946 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: WARNING py.warnings [None req-1f389025-ead3-444b-8ac5-5012fe3ef5c1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 18 05:12:52.825946 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: warnings.warn(msg) Jun 18 05:12:52.825946 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: Jun 18 05:12:52.826255 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f389025-ead3-444b-8ac5-5012fe3ef5c1 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:12:52.964324 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-1f389025-ead3-444b-8ac5-5012fe3ef5c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7eef79de2b9745da908815751a816904', 'request_id': 'req-1f389025-ead3-444b-8ac5-5012fe3ef5c1'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:12:52.970699 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-1f389025-ead3-444b-8ac5-5012fe3ef5c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7eef79de2b9745da908815751a816904', 'request_id': 'req-1f389025-ead3-444b-8ac5-5012fe3ef5c1'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:12:53.048301 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-1f389025-ead3-444b-8ac5-5012fe3ef5c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7eef79de2b9745da908815751a816904', 'request_id': 'req-1f389025-ead3-444b-8ac5-5012fe3ef5c1'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:12:53.067416 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-1f389025-ead3-444b-8ac5-5012fe3ef5c1 admin admin] The token cache is being invalidate because project 7eef79de2b9745da908815751a816904 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80468) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 18 05:12:53.067705 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-1f389025-ead3-444b-8ac5-5012fe3ef5c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:12:53.068410 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-1f389025-ead3-444b-8ac5-5012fe3ef5c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:12:53.070188 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 745/1510] 10.222.0.23 () {58 vars in 1232 bytes} [Sat Jun 18 05:12:52 2022] DELETE /identity/v3/projects/7eef79de2b9745da908815751a816904 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:13:02.332811 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=29 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:13:03.064341 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:14:29.724482 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20213c5-c8b3-4f9d-bc88-3fd40a45b1a2 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:14:29.725129 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20213c5-c8b3-4f9d-bc88-3fd40a45b1a2 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:14:29.725129 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20213c5-c8b3-4f9d-bc88-3fd40a45b1a2 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:14:29.726457 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 766/1511] 127.0.0.1 () {54 vars in 901 bytes} [Sat Jun 18 05:14:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 05:14:29.731221 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73c5f93-7b6b-494a-8846-e33c38d93fb0 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:14:29.731629 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73c5f93-7b6b-494a-8846-e33c38d93fb0 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:14:29.731786 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73c5f93-7b6b-494a-8846-e33c38d93fb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:14:29.750659 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-f73c5f93-7b6b-494a-8846-e33c38d93fb0 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:14:29.755764 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-f73c5f93-7b6b-494a-8846-e33c38d93fb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:14:29.812095 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 746/1512] 127.0.0.1 () {58 vars in 998 bytes} [Sat Jun 18 05:14:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:14:29.973279 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20213c5-c8b3-4f9d-bc88-3fd40a45b1a2 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:14:29.984007 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-b20213c5-c8b3-4f9d-bc88-3fd40a45b1a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:14:30.052613 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20213c5-c8b3-4f9d-bc88-3fd40a45b1a2 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:14:30.058096 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293ffc00-b149-4ddd-a976-5f44f1177c47 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15e7f45ee0744c1d9602187eac8e8b80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:14:30.058734 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293ffc00-b149-4ddd-a976-5f44f1177c47 service nova] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:14:30.059017 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293ffc00-b149-4ddd-a976-5f44f1177c47 service nova] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:14:30.059295 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293ffc00-b149-4ddd-a976-5f44f1177c47 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:14:30.067426 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293ffc00-b149-4ddd-a976-5f44f1177c47 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:14:30.069735 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293ffc00-b149-4ddd-a976-5f44f1177c47 service nova] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:14:30.079114 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 767/1513] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:14:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:14:30.914307 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c18553f-e1c4-4b35-ad2c-ce75fea55a5e None None] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:14:30.914871 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c18553f-e1c4-4b35-ad2c-ce75fea55a5e None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:14:30.914871 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c18553f-e1c4-4b35-ad2c-ce75fea55a5e None None] PATH_INFO: `/` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:14:30.916222 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 747/1514] 127.0.0.1 () {54 vars in 901 bytes} [Sat Jun 18 05:14:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 05:14:30.921066 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5911ac66-cf70-4cdb-b4e6-04ce1dcbee42 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:14:30.921468 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5911ac66-cf70-4cdb-b4e6-04ce1dcbee42 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:14:30.921648 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5911ac66-cf70-4cdb-b4e6-04ce1dcbee42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:14:30.940820 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-5911ac66-cf70-4cdb-b4e6-04ce1dcbee42 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:14:30.942169 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-5911ac66-cf70-4cdb-b4e6-04ce1dcbee42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:14:30.950296 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 768/1515] 127.0.0.1 () {58 vars in 998 bytes} [Sat Jun 18 05:14:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:14:31.103920 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c18553f-e1c4-4b35-ad2c-ce75fea55a5e None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:14:31.118829 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c18553f-e1c4-4b35-ad2c-ce75fea55a5e None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:14:31.124348 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51ce39f-d284-4ad8-8aa8-cfee600f3c25 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15e7f45ee0744c1d9602187eac8e8b80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:14:31.124981 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51ce39f-d284-4ad8-8aa8-cfee600f3c25 service nova] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:14:31.125271 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51ce39f-d284-4ad8-8aa8-cfee600f3c25 service nova] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:14:31.125579 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51ce39f-d284-4ad8-8aa8-cfee600f3c25 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:14:31.134056 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51ce39f-d284-4ad8-8aa8-cfee600f3c25 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:14:31.135708 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51ce39f-d284-4ad8-8aa8-cfee600f3c25 service nova] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:14:31.145529 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 748/1516] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:14:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:14:31.226183 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607be544-1552-4fa4-92cf-6ec2d6a10144 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:14:31.226616 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607be544-1552-4fa4-92cf-6ec2d6a10144 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:14:31.227059 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607be544-1552-4fa4-92cf-6ec2d6a10144 None None] PATH_INFO: `/v3` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:14:31.228544 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 769/1517] 10.222.0.15 () {54 vars in 925 bytes} [Sat Jun 18 05:14:31 2022] GET /identity/v3 => generated 255 bytes in 4 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 05:14:31.234820 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4288116d-cc80-46d6-8fbf-831f9f95cb18 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:14:31.235113 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4288116d-cc80-46d6-8fbf-831f9f95cb18 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:14:31.235422 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4288116d-cc80-46d6-8fbf-831f9f95cb18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:14:31.254450 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-4288116d-cc80-46d6-8fbf-831f9f95cb18 None None] MFA Rules not processed for user `5323ffd91f864680b0c0084eb3f44d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:14:31.256116 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-4288116d-cc80-46d6-8fbf-831f9f95cb18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:14:31.319328 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 749/1518] 10.222.0.15 () {58 vars in 1015 bytes} [Sat Jun 18 05:14:31 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:14:40.430027 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2e390c-8a93-4d07-bc2f-30667fcd0d71 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:14:40.430517 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2e390c-8a93-4d07-bc2f-30667fcd0d71 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:14:40.430627 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2e390c-8a93-4d07-bc2f-30667fcd0d71 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:14:40.431950 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 770/1519] 10.222.0.23 () {54 vars in 960 bytes} [Sat Jun 18 05:14:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 05:14:40.436812 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068ccd9f-75c5-46f2-9486-74e8a1fd57a5 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:14:40.437193 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068ccd9f-75c5-46f2-9486-74e8a1fd57a5 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:14:40.437372 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068ccd9f-75c5-46f2-9486-74e8a1fd57a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:14:40.455693 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-068ccd9f-75c5-46f2-9486-74e8a1fd57a5 None None] MFA Rules not processed for user `5323ffd91f864680b0c0084eb3f44d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:14:40.456877 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-068ccd9f-75c5-46f2-9486-74e8a1fd57a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:14:40.465500 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 750/1520] 10.222.0.23 () {58 vars in 1057 bytes} [Sat Jun 18 05:14:40 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:14:40.472734 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2e390c-8a93-4d07-bc2f-30667fcd0d71 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:14:40.486157 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2e390c-8a93-4d07-bc2f-30667fcd0d71 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:14:40.491609 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0993026-b015-4d22-9169-b54413bcecdb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5323ffd91f864680b0c0084eb3f44d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:14:40.492231 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0993026-b015-4d22-9169-b54413bcecdb service placement] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:14:40.492514 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0993026-b015-4d22-9169-b54413bcecdb service placement] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:14:40.492790 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0993026-b015-4d22-9169-b54413bcecdb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:14:40.501038 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0993026-b015-4d22-9169-b54413bcecdb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:14:40.502692 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0993026-b015-4d22-9169-b54413bcecdb service placement] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:14:40.512879 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 771/1521] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:14:40 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:14:42.409802 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef89a0f-2b4b-4e02-90a7-988127e995d3 None None] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:14:42.410291 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef89a0f-2b4b-4e02-90a7-988127e995d3 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:14:42.410392 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef89a0f-2b4b-4e02-90a7-988127e995d3 None None] PATH_INFO: `/` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:14:42.411725 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 751/1522] 127.0.0.1 () {54 vars in 901 bytes} [Sat Jun 18 05:14:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 05:14:42.416726 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e692c4-61f6-431b-a6c3-dc7a7242745b None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:14:42.417104 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e692c4-61f6-431b-a6c3-dc7a7242745b None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:14:42.417278 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e692c4-61f6-431b-a6c3-dc7a7242745b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:14:42.435283 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-15e692c4-61f6-431b-a6c3-dc7a7242745b None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:14:42.436459 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-15e692c4-61f6-431b-a6c3-dc7a7242745b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:14:42.445229 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 772/1523] 127.0.0.1 () {58 vars in 998 bytes} [Sat Jun 18 05:14:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:14:42.601771 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef89a0f-2b4b-4e02-90a7-988127e995d3 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:14:42.617012 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef89a0f-2b4b-4e02-90a7-988127e995d3 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:14:42.622495 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454c3ac0-7d26-4965-b84e-f3eba4c5291d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15e7f45ee0744c1d9602187eac8e8b80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:14:42.623116 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454c3ac0-7d26-4965-b84e-f3eba4c5291d service nova] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:14:42.623461 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454c3ac0-7d26-4965-b84e-f3eba4c5291d service nova] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:14:42.623736 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454c3ac0-7d26-4965-b84e-f3eba4c5291d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:14:42.634640 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454c3ac0-7d26-4965-b84e-f3eba4c5291d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:14:42.636209 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454c3ac0-7d26-4965-b84e-f3eba4c5291d service nova] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:14:42.648347 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 752/1524] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:14:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:14:43.465944 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b40f36-a24a-4b4c-bab2-5cf42fe551c5 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:14:43.466441 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b40f36-a24a-4b4c-bab2-5cf42fe551c5 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:14:43.466557 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b40f36-a24a-4b4c-bab2-5cf42fe551c5 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:14:43.467880 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 773/1525] 127.0.0.1 () {54 vars in 901 bytes} [Sat Jun 18 05:14:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 05:14:43.472613 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a6f54f-5340-4ca2-85b1-caea186fcbdd None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:14:43.472996 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a6f54f-5340-4ca2-85b1-caea186fcbdd None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:14:43.473172 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a6f54f-5340-4ca2-85b1-caea186fcbdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:14:43.491457 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-a7a6f54f-5340-4ca2-85b1-caea186fcbdd None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:14:43.492656 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-a7a6f54f-5340-4ca2-85b1-caea186fcbdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:14:43.500617 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 753/1526] 127.0.0.1 () {58 vars in 998 bytes} [Sat Jun 18 05:14:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:14:43.653286 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b40f36-a24a-4b4c-bab2-5cf42fe551c5 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:14:43.667677 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b40f36-a24a-4b4c-bab2-5cf42fe551c5 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:14:43.673130 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471b6fca-cd54-4dcd-8238-de18b78230dd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15e7f45ee0744c1d9602187eac8e8b80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:14:43.673798 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471b6fca-cd54-4dcd-8238-de18b78230dd service nova] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:14:43.674088 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471b6fca-cd54-4dcd-8238-de18b78230dd service nova] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:14:43.674367 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471b6fca-cd54-4dcd-8238-de18b78230dd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:14:43.682833 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471b6fca-cd54-4dcd-8238-de18b78230dd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:14:43.684456 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471b6fca-cd54-4dcd-8238-de18b78230dd service nova] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:14:43.694629 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 774/1527] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:14:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:14:47.441103 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deb6c62-edd0-4cc4-9aaa-b5937af5b242 None None] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:14:47.441725 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deb6c62-edd0-4cc4-9aaa-b5937af5b242 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:14:47.441844 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deb6c62-edd0-4cc4-9aaa-b5937af5b242 None None] PATH_INFO: `/` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:14:47.443124 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 754/1528] 127.0.0.1 () {54 vars in 916 bytes} [Sat Jun 18 05:14:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 18 05:14:47.449184 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6176c654-1797-43c9-a2bf-107916cd3b9c None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:14:47.452041 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6176c654-1797-43c9-a2bf-107916cd3b9c None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:14:47.452041 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6176c654-1797-43c9-a2bf-107916cd3b9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:14:47.468259 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-6176c654-1797-43c9-a2bf-107916cd3b9c None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:14:47.469486 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-6176c654-1797-43c9-a2bf-107916cd3b9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:14:47.477491 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 775/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Sat Jun 18 05:14:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:14:47.513922 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8deb6c62-edd0-4cc4-9aaa-b5937af5b242 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:14:47.515540 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-8deb6c62-edd0-4cc4-9aaa-b5937af5b242 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:14:47.573767 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8deb6c62-edd0-4cc4-9aaa-b5937af5b242 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:14:47.577309 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ab2d0a-571f-4a13-98d9-940b53da0900 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c4d7cde8f4ad9aa479bb09e325db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:14:47.577975 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ab2d0a-571f-4a13-98d9-940b53da0900 service neutron] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:14:47.578267 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ab2d0a-571f-4a13-98d9-940b53da0900 service neutron] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:14:47.578547 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ab2d0a-571f-4a13-98d9-940b53da0900 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:14:47.587442 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ab2d0a-571f-4a13-98d9-940b53da0900 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:14:47.588878 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ab2d0a-571f-4a13-98d9-940b53da0900 service neutron] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:14:47.593930 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:14:47.598690 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 755/1530] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:14:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:14:49.626900 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8269c2ae-1463-40f7-94d3-79d777b59b94 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:14:49.627441 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8269c2ae-1463-40f7-94d3-79d777b59b94 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:14:49.627441 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8269c2ae-1463-40f7-94d3-79d777b59b94 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:14:49.628818 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 776/1531] 127.0.0.1 () {54 vars in 916 bytes} [Sat Jun 18 05:14:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 05:14:49.634883 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be4a396-b699-4139-ad13-59963a565255 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:14:49.635293 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be4a396-b699-4139-ad13-59963a565255 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:14:49.635444 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be4a396-b699-4139-ad13-59963a565255 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:14:49.654070 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-0be4a396-b699-4139-ad13-59963a565255 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:14:49.655279 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-0be4a396-b699-4139-ad13-59963a565255 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:14:49.663065 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 756/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Sat Jun 18 05:14:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:14:49.699601 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8269c2ae-1463-40f7-94d3-79d777b59b94 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:14:49.711500 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8269c2ae-1463-40f7-94d3-79d777b59b94 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:14:49.719181 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51cf915-cc76-49b7-8464-1cbd720df0e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c4d7cde8f4ad9aa479bb09e325db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:14:49.719181 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51cf915-cc76-49b7-8464-1cbd720df0e2 service neutron] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:14:49.719181 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51cf915-cc76-49b7-8464-1cbd720df0e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:14:49.719181 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51cf915-cc76-49b7-8464-1cbd720df0e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:14:49.725594 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51cf915-cc76-49b7-8464-1cbd720df0e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:14:49.727023 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51cf915-cc76-49b7-8464-1cbd720df0e2 service neutron] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:14:49.736482 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 777/1533] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:14:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:14:53.418381 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa30a4c8-ca17-45cc-a781-fb57d7b29c78 None None] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:14:53.418881 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa30a4c8-ca17-45cc-a781-fb57d7b29c78 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:14:53.419005 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa30a4c8-ca17-45cc-a781-fb57d7b29c78 None None] PATH_INFO: `/` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:14:53.420276 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 757/1534] 127.0.0.1 () {54 vars in 916 bytes} [Sat Jun 18 05:14:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 05:14:53.425518 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa60aabd-a07c-4aae-8854-ac357dc45a88 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:14:53.425929 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa60aabd-a07c-4aae-8854-ac357dc45a88 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:14:53.426096 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa60aabd-a07c-4aae-8854-ac357dc45a88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:14:53.433044 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=62 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:14:53.445559 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-aa60aabd-a07c-4aae-8854-ac357dc45a88 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:14:53.446756 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-aa60aabd-a07c-4aae-8854-ac357dc45a88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:14:53.454704 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 778/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Sat Jun 18 05:14:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:14:53.755268 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa30a4c8-ca17-45cc-a781-fb57d7b29c78 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:14:53.757499 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-aa30a4c8-ca17-45cc-a781-fb57d7b29c78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:14:53.815098 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa30a4c8-ca17-45cc-a781-fb57d7b29c78 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:14:53.818532 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5be5b1b-4ef0-46e7-ab93-662b0e91b1ab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9de0428abc8400aa338aba4f7c34c94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:14:53.820286 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5be5b1b-4ef0-46e7-ab93-662b0e91b1ab service cinder] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:14:53.820560 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5be5b1b-4ef0-46e7-ab93-662b0e91b1ab service cinder] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:14:53.820833 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5be5b1b-4ef0-46e7-ab93-662b0e91b1ab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:14:53.829601 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5be5b1b-4ef0-46e7-ab93-662b0e91b1ab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:14:53.831015 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5be5b1b-4ef0-46e7-ab93-662b0e91b1ab service cinder] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:14:53.840904 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 758/1536] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:14:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 18 05:14:55.908071 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e902d00-3dbe-4fed-889c-e2372f1a7dfb None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:14:55.908645 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e902d00-3dbe-4fed-889c-e2372f1a7dfb None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:14:55.908645 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e902d00-3dbe-4fed-889c-e2372f1a7dfb None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:14:55.910003 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 779/1537] 127.0.0.1 () {54 vars in 916 bytes} [Sat Jun 18 05:14:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 05:14:55.915753 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d888a93a-66e4-41fb-8dae-80fb526a9b8c None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:14:55.916152 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d888a93a-66e4-41fb-8dae-80fb526a9b8c None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:14:55.916303 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d888a93a-66e4-41fb-8dae-80fb526a9b8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:14:55.936215 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-d888a93a-66e4-41fb-8dae-80fb526a9b8c None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:14:55.937434 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-d888a93a-66e4-41fb-8dae-80fb526a9b8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:14:55.945788 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 759/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Sat Jun 18 05:14:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:14:56.253716 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e902d00-3dbe-4fed-889c-e2372f1a7dfb None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:14:56.255412 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-0e902d00-3dbe-4fed-889c-e2372f1a7dfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:14:56.268595 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e902d00-3dbe-4fed-889c-e2372f1a7dfb None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:14:56.272900 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55708d5b-4399-4911-ad3c-7593c3206adc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9de0428abc8400aa338aba4f7c34c94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:14:56.273703 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55708d5b-4399-4911-ad3c-7593c3206adc service cinder] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:14:56.273986 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55708d5b-4399-4911-ad3c-7593c3206adc service cinder] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:14:56.274261 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55708d5b-4399-4911-ad3c-7593c3206adc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:14:56.283406 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55708d5b-4399-4911-ad3c-7593c3206adc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:14:56.284895 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55708d5b-4399-4911-ad3c-7593c3206adc service cinder] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:14:56.294821 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 780/1539] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:14:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:15:05.930080 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=32 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:15:06.290696 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=8 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:16:23.787578 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a8d9a7-0336-428f-ad64-4fa2ef9ed137 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:23.788406 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a8d9a7-0336-428f-ad64-4fa2ef9ed137 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:23.788406 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a8d9a7-0336-428f-ad64-4fa2ef9ed137 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:23.813483 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-68a8d9a7-0336-428f-ad64-4fa2ef9ed137 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:16:23.814740 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-68a8d9a7-0336-428f-ad64-4fa2ef9ed137 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:16:23.828370 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 760/1540] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:16:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:16:23.840102 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221f0938-0ed6-42e7-9d7b-066ba1162def None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:23.840689 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221f0938-0ed6-42e7-9d7b-066ba1162def None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:23.840689 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221f0938-0ed6-42e7-9d7b-066ba1162def None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:23.851731 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d428fff-faab-44d4-8d54-9a9989c2e7c9 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:23.852019 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d428fff-faab-44d4-8d54-9a9989c2e7c9 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:23.852491 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d428fff-faab-44d4-8d54-9a9989c2e7c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:23.864866 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-221f0938-0ed6-42e7-9d7b-066ba1162def None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:16:23.870650 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-221f0938-0ed6-42e7-9d7b-066ba1162def None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:16:23.873586 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-9d428fff-faab-44d4-8d54-9a9989c2e7c9 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:16:23.875261 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-9d428fff-faab-44d4-8d54-9a9989c2e7c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:16:23.887642 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 761/1541] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:16:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 18 05:16:23.890731 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2076fe0-d44b-4501-bce3-5d1181133005 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:23.891182 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 781/1542] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:16:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:16:23.891792 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2076fe0-d44b-4501-bce3-5d1181133005 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:23.892374 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2076fe0-d44b-4501-bce3-5d1181133005 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:23.894908 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221f0938-0ed6-42e7-9d7b-066ba1162def None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:23.910882 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221f0938-0ed6-42e7-9d7b-066ba1162def None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:23.924454 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7132d28f-b95c-4723-897e-9e44cd073342 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:23.929475 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7132d28f-b95c-4723-897e-9e44cd073342 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:23.929776 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7132d28f-b95c-4723-897e-9e44cd073342 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:23.930060 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7132d28f-b95c-4723-897e-9e44cd073342 admin admin] PATH_INFO: `/v3/domains` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:23.930654 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-d2076fe0-d44b-4501-bce3-5d1181133005 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:16:23.930787 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7132d28f-b95c-4723-897e-9e44cd073342 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 05:16:23.931101 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7132d28f-b95c-4723-897e-9e44cd073342 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:23.932028 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-d2076fe0-d44b-4501-bce3-5d1181133005 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:16:23.932160 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: WARNING py.warnings [None req-7132d28f-b95c-4723-897e-9e44cd073342 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 18 05:16:23.932160 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: warnings.warn(msg) Jun 18 05:16:23.932160 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: Jun 18 05:16:23.933477 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7132d28f-b95c-4723-897e-9e44cd073342 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:23.948633 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 782/1543] 10.222.0.23 () {58 vars in 1152 bytes} [Sat Jun 18 05:16:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 05:16:23.949875 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 762/1544] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:16:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:16:23.950026 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff249c6-b22c-469b-a4a2-565fd2438eda None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:23.950294 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff249c6-b22c-469b-a4a2-565fd2438eda None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:23.950559 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff249c6-b22c-469b-a4a2-565fd2438eda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:23.958450 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2076fe0-d44b-4501-bce3-5d1181133005 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:23.980359 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2076fe0-d44b-4501-bce3-5d1181133005 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:23.984207 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-9ff249c6-b22c-469b-a4a2-565fd2438eda None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:16:23.985538 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-9ff249c6-b22c-469b-a4a2-565fd2438eda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:16:23.989912 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca98565-4c3a-4fe5-9f4e-df24ce7c0d46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:23.990605 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca98565-4c3a-4fe5-9f4e-df24ce7c0d46 admin admin] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:23.991425 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca98565-4c3a-4fe5-9f4e-df24ce7c0d46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:23.991893 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca98565-4c3a-4fe5-9f4e-df24ce7c0d46 admin admin] PATH_INFO: `/v3/projects` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:23.992618 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca98565-4c3a-4fe5-9f4e-df24ce7c0d46 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:23.996316 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: WARNING py.warnings [None req-1ca98565-4c3a-4fe5-9f4e-df24ce7c0d46 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 18 05:16:23.996316 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: warnings.warn(msg) Jun 18 05:16:23.996316 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: Jun 18 05:16:23.996316 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca98565-4c3a-4fe5-9f4e-df24ce7c0d46 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:24.007783 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 783/1545] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:16:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:16:24.009876 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc049281-8480-4b24-9f60-0298865d79f9 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:24.010168 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc049281-8480-4b24-9f60-0298865d79f9 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:24.010451 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc049281-8480-4b24-9f60-0298865d79f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:24.045805 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-fc049281-8480-4b24-9f60-0298865d79f9 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:16:24.045805 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-fc049281-8480-4b24-9f60-0298865d79f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:16:24.052239 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 784/1546] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:16:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:16:24.053800 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d07502-b068-4bd3-ab8b-911ed1ffc8a7 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:24.054261 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d07502-b068-4bd3-ab8b-911ed1ffc8a7 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:24.054549 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d07502-b068-4bd3-ab8b-911ed1ffc8a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:24.074624 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-67d07502-b068-4bd3-ab8b-911ed1ffc8a7 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:16:24.075844 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-67d07502-b068-4bd3-ab8b-911ed1ffc8a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:16:24.084235 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 785/1547] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:16:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:16:24.085449 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d07502-b068-4bd3-ab8b-911ed1ffc8a7 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:24.095948 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d07502-b068-4bd3-ab8b-911ed1ffc8a7 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:24.100614 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53462121-e1d2-49ca-9a0c-01fc79ba3780 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:24.101283 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53462121-e1d2-49ca-9a0c-01fc79ba3780 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:24.101654 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53462121-e1d2-49ca-9a0c-01fc79ba3780 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:24.102175 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53462121-e1d2-49ca-9a0c-01fc79ba3780 admin admin] PATH_INFO: `/v3/domains` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:24.102979 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53462121-e1d2-49ca-9a0c-01fc79ba3780 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 05:16:24.103296 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53462121-e1d2-49ca-9a0c-01fc79ba3780 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:24.104916 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53462121-e1d2-49ca-9a0c-01fc79ba3780 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:24.115915 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 763/1548] 10.222.0.23 () {60 vars in 1152 bytes} [Sat Jun 18 05:16:23 2022] POST /identity/v3/projects => generated 405 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:16:24.117764 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c191032-20f0-412d-85fe-3198f1234c12 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:24.117886 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 786/1549] 10.222.0.23 () {58 vars in 1152 bytes} [Sat Jun 18 05:16:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 05:16:24.118350 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c191032-20f0-412d-85fe-3198f1234c12 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:24.118780 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c191032-20f0-412d-85fe-3198f1234c12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:24.119312 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38684b68-17ef-4581-82d9-9c921bc92ea7 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:24.119607 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38684b68-17ef-4581-82d9-9c921bc92ea7 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:24.119882 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38684b68-17ef-4581-82d9-9c921bc92ea7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:24.142960 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-0c191032-20f0-412d-85fe-3198f1234c12 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:16:24.144177 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-0c191032-20f0-412d-85fe-3198f1234c12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:16:24.149854 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-38684b68-17ef-4581-82d9-9c921bc92ea7 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:16:24.151072 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-38684b68-17ef-4581-82d9-9c921bc92ea7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:16:24.212120 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 764/1550] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:16:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:16:24.213044 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c191032-20f0-412d-85fe-3198f1234c12 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:24.218826 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 787/1551] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:16:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:16:24.220383 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67988c2-edbe-45d8-97ee-d57bc8a72ac3 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:24.220900 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67988c2-edbe-45d8-97ee-d57bc8a72ac3 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:24.221183 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67988c2-edbe-45d8-97ee-d57bc8a72ac3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:24.227610 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c191032-20f0-412d-85fe-3198f1234c12 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:24.232295 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d32b855-4c5c-452c-8f1f-13359b0b5ade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:24.232954 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d32b855-4c5c-452c-8f1f-13359b0b5ade admin admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:24.233233 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d32b855-4c5c-452c-8f1f-13359b0b5ade admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:24.233553 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d32b855-4c5c-452c-8f1f-13359b0b5ade admin admin] PATH_INFO: `/v3/domains` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:24.234278 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d32b855-4c5c-452c-8f1f-13359b0b5ade admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 05:16:24.234588 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d32b855-4c5c-452c-8f1f-13359b0b5ade admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:24.236189 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d32b855-4c5c-452c-8f1f-13359b0b5ade admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:24.240532 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-c67988c2-edbe-45d8-97ee-d57bc8a72ac3 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:16:24.242407 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-c67988c2-edbe-45d8-97ee-d57bc8a72ac3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:16:24.253998 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 765/1552] 10.222.0.23 () {58 vars in 1152 bytes} [Sat Jun 18 05:16:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 05:16:24.253998 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d32b855-4c5c-452c-8f1f-13359b0b5ade admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:24.258814 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 788/1553] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:16:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:16:24.258814 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67988c2-edbe-45d8-97ee-d57bc8a72ac3 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:24.263725 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d32b855-4c5c-452c-8f1f-13359b0b5ade admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:24.265524 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67988c2-edbe-45d8-97ee-d57bc8a72ac3 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:24.268305 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bae49c-fb4d-4d22-820a-ba8d8106bc69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:24.268978 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bae49c-fb4d-4d22-820a-ba8d8106bc69 admin admin] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:24.269263 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bae49c-fb4d-4d22-820a-ba8d8106bc69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:24.269578 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bae49c-fb4d-4d22-820a-ba8d8106bc69 admin admin] PATH_INFO: `/v3/users` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:24.270092 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4412f04-e8b3-4f2f-9bcf-dd565cfa980e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:24.270272 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bae49c-fb4d-4d22-820a-ba8d8106bc69 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:24.270727 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4412f04-e8b3-4f2f-9bcf-dd565cfa980e admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:24.271011 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4412f04-e8b3-4f2f-9bcf-dd565cfa980e admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:24.271290 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4412f04-e8b3-4f2f-9bcf-dd565cfa980e admin admin] PATH_INFO: `/v3/projects` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:24.271975 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4412f04-e8b3-4f2f-9bcf-dd565cfa980e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:24.272203 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bae49c-fb4d-4d22-820a-ba8d8106bc69 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:24.274255 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4412f04-e8b3-4f2f-9bcf-dd565cfa980e admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:24.368668 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 766/1554] 10.222.0.23 () {60 vars in 1143 bytes} [Sat Jun 18 05:16:24 2022] POST /identity/v3/users => generated 303 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:16:24.369902 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc2b980-fc96-4c66-bdc6-9627352a64ba None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:24.370198 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc2b980-fc96-4c66-bdc6-9627352a64ba None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:24.370482 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc2b980-fc96-4c66-bdc6-9627352a64ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:24.388385 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-3bc2b980-fc96-4c66-bdc6-9627352a64ba None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:16:24.389590 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-3bc2b980-fc96-4c66-bdc6-9627352a64ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:16:24.400212 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 789/1555] 10.222.0.23 () {60 vars in 1152 bytes} [Sat Jun 18 05:16:24 2022] POST /identity/v3/projects => generated 379 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:16:24.400212 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8b5ffe-a00d-4a48-93d4-b783b6625c58 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:24.400212 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8b5ffe-a00d-4a48-93d4-b783b6625c58 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:24.400212 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8b5ffe-a00d-4a48-93d4-b783b6625c58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:24.414009 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-bc8b5ffe-a00d-4a48-93d4-b783b6625c58 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:16:24.415256 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-bc8b5ffe-a00d-4a48-93d4-b783b6625c58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:16:24.466154 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 767/1556] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:16:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:16:24.467627 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc17360-40e3-49cb-a8ba-46adebb38e06 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:24.467918 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc17360-40e3-49cb-a8ba-46adebb38e06 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:24.468193 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc17360-40e3-49cb-a8ba-46adebb38e06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:24.485364 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 790/1557] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:16:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:16:24.486451 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-2bc17360-40e3-49cb-a8ba-46adebb38e06 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:16:24.486743 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3483ebdf-dcb6-4614-b3df-06ce46381128 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:24.487037 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3483ebdf-dcb6-4614-b3df-06ce46381128 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:24.487313 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3483ebdf-dcb6-4614-b3df-06ce46381128 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:24.487703 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-2bc17360-40e3-49cb-a8ba-46adebb38e06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:16:24.500196 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 768/1558] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:16:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:16:24.500196 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc17360-40e3-49cb-a8ba-46adebb38e06 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:24.505983 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-3483ebdf-dcb6-4614-b3df-06ce46381128 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:16:24.507774 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-3483ebdf-dcb6-4614-b3df-06ce46381128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:16:24.510889 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc17360-40e3-49cb-a8ba-46adebb38e06 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:24.515493 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc022099-f42e-4e3b-99e2-8a347dd57e1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:24.516143 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc022099-f42e-4e3b-99e2-8a347dd57e1d admin admin] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:24.516251 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 791/1559] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:16:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:16:24.516438 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc022099-f42e-4e3b-99e2-8a347dd57e1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:24.516705 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc022099-f42e-4e3b-99e2-8a347dd57e1d admin admin] PATH_INFO: `/v3/projects` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:24.517556 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc022099-f42e-4e3b-99e2-8a347dd57e1d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:24.519465 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc022099-f42e-4e3b-99e2-8a347dd57e1d admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:24.521571 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3483ebdf-dcb6-4614-b3df-06ce46381128 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:24.531569 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3483ebdf-dcb6-4614-b3df-06ce46381128 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:24.536468 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88259b3f-1a47-44ac-ae16-4a10bcef460f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:24.537169 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88259b3f-1a47-44ac-ae16-4a10bcef460f admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:24.537489 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88259b3f-1a47-44ac-ae16-4a10bcef460f admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:24.537766 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88259b3f-1a47-44ac-ae16-4a10bcef460f admin admin] PATH_INFO: `/v3/domains` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:24.539041 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88259b3f-1a47-44ac-ae16-4a10bcef460f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 05:16:24.539352 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88259b3f-1a47-44ac-ae16-4a10bcef460f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:24.540971 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88259b3f-1a47-44ac-ae16-4a10bcef460f admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:24.549777 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:16:24.555117 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 792/1560] 10.222.0.23 () {58 vars in 1152 bytes} [Sat Jun 18 05:16:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 05:16:24.555117 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88259b3f-1a47-44ac-ae16-4a10bcef460f admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:24.564632 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88259b3f-1a47-44ac-ae16-4a10bcef460f admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:24.570014 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04e9b04-530f-4e6d-bf01-12fd6f6332cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:24.570648 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04e9b04-530f-4e6d-bf01-12fd6f6332cb admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:24.570926 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04e9b04-530f-4e6d-bf01-12fd6f6332cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:24.571198 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04e9b04-530f-4e6d-bf01-12fd6f6332cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:24.571903 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04e9b04-530f-4e6d-bf01-12fd6f6332cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:24.573592 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04e9b04-530f-4e6d-bf01-12fd6f6332cb admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:24.583770 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 793/1561] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:16:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:16:24.589715 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04e9b04-530f-4e6d-bf01-12fd6f6332cb admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:24.594552 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04e9b04-530f-4e6d-bf01-12fd6f6332cb admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:24.599001 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce0623e-93b5-4f9b-86ab-a6d8dd76ef63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:24.599613 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce0623e-93b5-4f9b-86ab-a6d8dd76ef63 admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:24.599891 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce0623e-93b5-4f9b-86ab-a6d8dd76ef63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:24.600167 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce0623e-93b5-4f9b-86ab-a6d8dd76ef63 admin admin] PATH_INFO: `/v3/users` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:24.601344 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce0623e-93b5-4f9b-86ab-a6d8dd76ef63 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:24.602365 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: WARNING py.warnings [None req-fce0623e-93b5-4f9b-86ab-a6d8dd76ef63 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 18 05:16:24.602365 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: warnings.warn(msg) Jun 18 05:16:24.602365 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: Jun 18 05:16:24.603548 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce0623e-93b5-4f9b-86ab-a6d8dd76ef63 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:24.609288 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:16:24.617425 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 769/1562] 10.222.0.23 () {60 vars in 1152 bytes} [Sat Jun 18 05:16:24 2022] POST /identity/v3/projects => generated 405 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:16:24.618202 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855a36d3-a054-4a53-a0d1-7e348bbf9ab2 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:24.618202 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855a36d3-a054-4a53-a0d1-7e348bbf9ab2 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:24.618375 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855a36d3-a054-4a53-a0d1-7e348bbf9ab2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:24.636959 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-855a36d3-a054-4a53-a0d1-7e348bbf9ab2 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:16:24.638279 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-855a36d3-a054-4a53-a0d1-7e348bbf9ab2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:16:24.696373 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 770/1563] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:16:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:16:24.697488 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e13f209-6af5-407f-97ba-cb50a78b759c None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:24.697783 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e13f209-6af5-407f-97ba-cb50a78b759c None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:24.698063 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e13f209-6af5-407f-97ba-cb50a78b759c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:24.701294 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 794/1564] 10.222.0.23 () {60 vars in 1143 bytes} [Sat Jun 18 05:16:24 2022] POST /identity/v3/users => generated 290 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:16:24.702551 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce0623e-93b5-4f9b-86ab-a6d8dd76ef63 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:24.713131 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce0623e-93b5-4f9b-86ab-a6d8dd76ef63 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:24.716596 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-1e13f209-6af5-407f-97ba-cb50a78b759c None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:16:24.717846 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-1e13f209-6af5-407f-97ba-cb50a78b759c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:16:24.717988 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4692e275-c290-4891-bed3-b44d35ce55ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:24.718630 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4692e275-c290-4891-bed3-b44d35ce55ba admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:24.718910 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4692e275-c290-4891-bed3-b44d35ce55ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:24.719183 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4692e275-c290-4891-bed3-b44d35ce55ba admin admin] PATH_INFO: `/v3/domains` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:24.719899 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4692e275-c290-4891-bed3-b44d35ce55ba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 05:16:24.720202 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4692e275-c290-4891-bed3-b44d35ce55ba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:24.721887 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4692e275-c290-4891-bed3-b44d35ce55ba admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:24.730270 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 771/1565] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:16:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:16:24.730270 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e13f209-6af5-407f-97ba-cb50a78b759c None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:24.743906 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 795/1566] 10.222.0.23 () {58 vars in 1152 bytes} [Sat Jun 18 05:16:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 05:16:24.743906 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4692e275-c290-4891-bed3-b44d35ce55ba admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:24.747911 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e13f209-6af5-407f-97ba-cb50a78b759c None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:24.747911 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0565c0-3dc5-4171-b447-277e823417af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:24.747911 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0565c0-3dc5-4171-b447-277e823417af admin admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:24.747911 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0565c0-3dc5-4171-b447-277e823417af admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:24.748285 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0565c0-3dc5-4171-b447-277e823417af admin admin] PATH_INFO: `/v3/domains` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:24.748285 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0565c0-3dc5-4171-b447-277e823417af admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 05:16:24.748285 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0565c0-3dc5-4171-b447-277e823417af admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:24.748285 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0565c0-3dc5-4171-b447-277e823417af admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:24.749026 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4692e275-c290-4891-bed3-b44d35ce55ba admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:24.753554 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13355947-270a-48f4-86b2-21621ed3891f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:24.754186 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13355947-270a-48f4-86b2-21621ed3891f admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:24.754463 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13355947-270a-48f4-86b2-21621ed3891f admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:24.754732 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13355947-270a-48f4-86b2-21621ed3891f admin admin] PATH_INFO: `/v3/projects` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:24.755407 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13355947-270a-48f4-86b2-21621ed3891f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:24.758455 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13355947-270a-48f4-86b2-21621ed3891f admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:24.759585 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 772/1567] 10.222.0.23 () {58 vars in 1152 bytes} [Sat Jun 18 05:16:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 05:16:24.759585 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0565c0-3dc5-4171-b447-277e823417af admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:24.769847 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0565c0-3dc5-4171-b447-277e823417af admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:24.775228 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296ac5b4-e16d-4cae-a160-c531498afccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:24.776323 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296ac5b4-e16d-4cae-a160-c531498afccc admin admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:24.776323 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296ac5b4-e16d-4cae-a160-c531498afccc admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:24.776652 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296ac5b4-e16d-4cae-a160-c531498afccc admin admin] PATH_INFO: `/v3/roles` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:24.777500 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296ac5b4-e16d-4cae-a160-c531498afccc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:24.779080 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296ac5b4-e16d-4cae-a160-c531498afccc admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:24.790412 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 773/1568] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:16:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:16:24.791177 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296ac5b4-e16d-4cae-a160-c531498afccc admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:24.801601 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296ac5b4-e16d-4cae-a160-c531498afccc admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:24.806615 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411b39a8-e54f-49b0-9eeb-7cf9dcb84306 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:24.807240 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411b39a8-e54f-49b0-9eeb-7cf9dcb84306 admin admin] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:24.807574 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411b39a8-e54f-49b0-9eeb-7cf9dcb84306 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:24.807837 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411b39a8-e54f-49b0-9eeb-7cf9dcb84306 admin admin] PATH_INFO: `/v3/users` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:24.808511 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411b39a8-e54f-49b0-9eeb-7cf9dcb84306 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:24.810439 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411b39a8-e54f-49b0-9eeb-7cf9dcb84306 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:24.876539 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 796/1569] 10.222.0.23 () {60 vars in 1152 bytes} [Sat Jun 18 05:16:24 2022] POST /identity/v3/projects => generated 367 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:16:24.877660 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13355947-270a-48f4-86b2-21621ed3891f admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:24.933986 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 774/1570] 10.222.0.23 () {60 vars in 1143 bytes} [Sat Jun 18 05:16:24 2022] POST /identity/v3/users => generated 303 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:16:24.938001 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411b39a8-e54f-49b0-9eeb-7cf9dcb84306 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:24.939141 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13355947-270a-48f4-86b2-21621ed3891f admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:24.944815 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca0ac48-44cd-44aa-92e1-57e50fbd4fbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:24.945516 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca0ac48-44cd-44aa-92e1-57e50fbd4fbb admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:24.945808 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca0ac48-44cd-44aa-92e1-57e50fbd4fbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:24.946092 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca0ac48-44cd-44aa-92e1-57e50fbd4fbb admin admin] PATH_INFO: `/v3/domains` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:24.946820 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca0ac48-44cd-44aa-92e1-57e50fbd4fbb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 05:16:24.947136 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca0ac48-44cd-44aa-92e1-57e50fbd4fbb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:24.948350 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411b39a8-e54f-49b0-9eeb-7cf9dcb84306 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:24.948769 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca0ac48-44cd-44aa-92e1-57e50fbd4fbb admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:24.952936 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d516f5c-b1d0-4372-a8b0-abd67829ca3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:24.953622 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d516f5c-b1d0-4372-a8b0-abd67829ca3f admin admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:24.953910 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d516f5c-b1d0-4372-a8b0-abd67829ca3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:24.954187 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d516f5c-b1d0-4372-a8b0-abd67829ca3f admin admin] PATH_INFO: `/v3/roles` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:24.954901 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d516f5c-b1d0-4372-a8b0-abd67829ca3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:24.956467 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d516f5c-b1d0-4372-a8b0-abd67829ca3f admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:24.961859 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 797/1571] 10.222.0.23 () {58 vars in 1152 bytes} [Sat Jun 18 05:16:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 05:16:24.975136 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca0ac48-44cd-44aa-92e1-57e50fbd4fbb admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:24.975136 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca0ac48-44cd-44aa-92e1-57e50fbd4fbb admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:24.975381 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 775/1572] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:16:24 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:16:24.975381 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d516f5c-b1d0-4372-a8b0-abd67829ca3f admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:24.980859 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fe3802-b9fc-40fa-8e37-5e314ee9d201 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:24.981576 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fe3802-b9fc-40fa-8e37-5e314ee9d201 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:24.981856 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fe3802-b9fc-40fa-8e37-5e314ee9d201 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:24.982125 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fe3802-b9fc-40fa-8e37-5e314ee9d201 admin admin] PATH_INFO: `/v3/domains` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:24.982870 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fe3802-b9fc-40fa-8e37-5e314ee9d201 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 05:16:24.983222 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fe3802-b9fc-40fa-8e37-5e314ee9d201 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:24.984809 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d516f5c-b1d0-4372-a8b0-abd67829ca3f admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:24.984932 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fe3802-b9fc-40fa-8e37-5e314ee9d201 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:24.989324 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239a1b44-cb62-4d73-87c4-ef7b346106ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:24.989984 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239a1b44-cb62-4d73-87c4-ef7b346106ad admin admin] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:24.990270 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239a1b44-cb62-4d73-87c4-ef7b346106ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:24.990543 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239a1b44-cb62-4d73-87c4-ef7b346106ad admin admin] PATH_INFO: `/v3/projects` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:24.991249 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239a1b44-cb62-4d73-87c4-ef7b346106ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:24.993107 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239a1b44-cb62-4d73-87c4-ef7b346106ad admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:24.997166 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 798/1573] 10.222.0.23 () {58 vars in 1152 bytes} [Sat Jun 18 05:16:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 05:16:24.998000 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fe3802-b9fc-40fa-8e37-5e314ee9d201 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:25.008475 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fe3802-b9fc-40fa-8e37-5e314ee9d201 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:25.013116 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36760ff2-c0fe-4e3a-91ca-def6570b9d23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:25.013798 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36760ff2-c0fe-4e3a-91ca-def6570b9d23 admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:25.014078 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36760ff2-c0fe-4e3a-91ca-def6570b9d23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:25.014353 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36760ff2-c0fe-4e3a-91ca-def6570b9d23 admin admin] PATH_INFO: `/v3/projects` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:25.015037 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36760ff2-c0fe-4e3a-91ca-def6570b9d23 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:25.016944 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36760ff2-c0fe-4e3a-91ca-def6570b9d23 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:25.107606 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 776/1574] 10.222.0.23 () {60 vars in 1152 bytes} [Sat Jun 18 05:16:24 2022] POST /identity/v3/projects => generated 389 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:16:25.107606 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239a1b44-cb62-4d73-87c4-ef7b346106ad admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:25.112984 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 799/1575] 10.222.0.23 () {60 vars in 1152 bytes} [Sat Jun 18 05:16:24 2022] POST /identity/v3/projects => generated 377 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:16:25.114049 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36760ff2-c0fe-4e3a-91ca-def6570b9d23 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:25.170827 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:16:25.172576 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239a1b44-cb62-4d73-87c4-ef7b346106ad admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:25.175967 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36760ff2-c0fe-4e3a-91ca-def6570b9d23 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:25.177522 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb82e4f-1217-4502-a3fa-2c32baddd571 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:25.178889 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb82e4f-1217-4502-a3fa-2c32baddd571 admin admin] REQUEST_METHOD: `PUT` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:25.179168 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb82e4f-1217-4502-a3fa-2c32baddd571 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:25.179643 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb82e4f-1217-4502-a3fa-2c32baddd571 admin admin] PATH_INFO: `/v3/projects/c7a2526885554e8d8db2b050e8e47d53/users/4568d73d43804b6c8119b89deece4e99/roles/cbf5c4c26898413d93e73b46b41046a3` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:25.181536 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d643e2-ddd7-457a-b7ad-7cabc31f6363 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:25.182208 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d643e2-ddd7-457a-b7ad-7cabc31f6363 admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:25.182489 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d643e2-ddd7-457a-b7ad-7cabc31f6363 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:25.182768 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d643e2-ddd7-457a-b7ad-7cabc31f6363 admin admin] PATH_INFO: `/v3/users` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:25.183457 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d643e2-ddd7-457a-b7ad-7cabc31f6363 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:25.185449 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d643e2-ddd7-457a-b7ad-7cabc31f6363 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:25.194725 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb82e4f-1217-4502-a3fa-2c32baddd571 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7a2526885554e8d8db2b050e8e47d53, user_id=4568d73d43804b6c8119b89deece4e99, role_id=cbf5c4c26898413d93e73b46b41046a3)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:25.196008 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: WARNING py.warnings [None req-ccb82e4f-1217-4502-a3fa-2c32baddd571 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 18 05:16:25.196008 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: warnings.warn(msg) Jun 18 05:16:25.196008 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: Jun 18 05:16:25.197935 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb82e4f-1217-4502-a3fa-2c32baddd571 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:25.299526 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 800/1576] 10.222.0.23 () {60 vars in 1143 bytes} [Sat Jun 18 05:16:25 2022] POST /identity/v3/users => generated 284 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:16:25.300348 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d643e2-ddd7-457a-b7ad-7cabc31f6363 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:25.310279 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d643e2-ddd7-457a-b7ad-7cabc31f6363 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:25.317091 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65dc554-67df-4051-94b6-36988725a9ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:25.317091 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65dc554-67df-4051-94b6-36988725a9ca admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:25.317091 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65dc554-67df-4051-94b6-36988725a9ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:25.317091 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65dc554-67df-4051-94b6-36988725a9ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:25.317488 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65dc554-67df-4051-94b6-36988725a9ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:25.318904 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65dc554-67df-4051-94b6-36988725a9ca admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:25.329661 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 801/1577] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:16:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:16:25.330480 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65dc554-67df-4051-94b6-36988725a9ca admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:25.340187 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65dc554-67df-4051-94b6-36988725a9ca admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:25.340717 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 777/1578] 10.222.0.23 () {60 vars in 1482 bytes} [Sat Jun 18 05:16:25 2022] PUT /identity/v3/projects/c7a2526885554e8d8db2b050e8e47d53/users/4568d73d43804b6c8119b89deece4e99/roles/cbf5c4c26898413d93e73b46b41046a3 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:16:25.341677 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb82e4f-1217-4502-a3fa-2c32baddd571 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:25.345762 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:16:25.398929 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb0dd19-3827-4ea2-b3b3-535af1a7606c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:25.399583 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb0dd19-3827-4ea2-b3b3-535af1a7606c admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:25.399865 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb0dd19-3827-4ea2-b3b3-535af1a7606c admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:25.400141 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb0dd19-3827-4ea2-b3b3-535af1a7606c admin admin] PATH_INFO: `/v3/projects` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:25.400836 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb0dd19-3827-4ea2-b3b3-535af1a7606c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:25.402870 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb0dd19-3827-4ea2-b3b3-535af1a7606c admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:25.408770 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb82e4f-1217-4502-a3fa-2c32baddd571 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:25.413748 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ba91c2-bbf2-4e29-af52-ee1cd30c9784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:25.414443 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ba91c2-bbf2-4e29-af52-ee1cd30c9784 admin admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:25.414731 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ba91c2-bbf2-4e29-af52-ee1cd30c9784 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:25.415006 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ba91c2-bbf2-4e29-af52-ee1cd30c9784 admin admin] PATH_INFO: `/v3/roles` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:25.415737 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ba91c2-bbf2-4e29-af52-ee1cd30c9784 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:25.417656 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ba91c2-bbf2-4e29-af52-ee1cd30c9784 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:25.428300 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 778/1579] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:16:25 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:16:25.430042 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ba91c2-bbf2-4e29-af52-ee1cd30c9784 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:25.439233 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ba91c2-bbf2-4e29-af52-ee1cd30c9784 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:25.443608 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8004fe3-ee76-4dd5-928c-f54e6ecefec3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:25.444341 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8004fe3-ee76-4dd5-928c-f54e6ecefec3 admin admin] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:25.444845 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8004fe3-ee76-4dd5-928c-f54e6ecefec3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:25.445125 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8004fe3-ee76-4dd5-928c-f54e6ecefec3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:25.445834 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8004fe3-ee76-4dd5-928c-f54e6ecefec3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:25.447673 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8004fe3-ee76-4dd5-928c-f54e6ecefec3 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:25.495038 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 802/1580] 10.222.0.23 () {60 vars in 1152 bytes} [Sat Jun 18 05:16:25 2022] POST /identity/v3/projects => generated 383 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:16:25.495777 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb0dd19-3827-4ea2-b3b3-535af1a7606c admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:25.558501 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb0dd19-3827-4ea2-b3b3-535af1a7606c admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:25.565034 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab853f3-5fc3-4f5c-ad95-70d5d32193ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:25.565762 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab853f3-5fc3-4f5c-ad95-70d5d32193ea admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:25.566052 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab853f3-5fc3-4f5c-ad95-70d5d32193ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:25.566350 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab853f3-5fc3-4f5c-ad95-70d5d32193ea admin admin] PATH_INFO: `/v3/users` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:25.567053 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab853f3-5fc3-4f5c-ad95-70d5d32193ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:25.568476 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 779/1581] 10.222.0.23 () {60 vars in 1152 bytes} [Sat Jun 18 05:16:25 2022] POST /identity/v3/projects => generated 357 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:16:25.569306 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8004fe3-ee76-4dd5-928c-f54e6ecefec3 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:25.569902 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab853f3-5fc3-4f5c-ad95-70d5d32193ea admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:25.633044 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8004fe3-ee76-4dd5-928c-f54e6ecefec3 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:25.637700 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f1c532-cd34-4479-b91d-86f82f7ff0ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:25.643870 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f1c532-cd34-4479-b91d-86f82f7ff0ec admin admin] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:25.643870 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f1c532-cd34-4479-b91d-86f82f7ff0ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:25.643870 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f1c532-cd34-4479-b91d-86f82f7ff0ec admin admin] PATH_INFO: `/v3/users` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:25.643870 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f1c532-cd34-4479-b91d-86f82f7ff0ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:25.643870 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f1c532-cd34-4479-b91d-86f82f7ff0ec admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:25.666220 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 803/1582] 10.222.0.23 () {60 vars in 1143 bytes} [Sat Jun 18 05:16:25 2022] POST /identity/v3/users => generated 295 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:16:25.667059 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab853f3-5fc3-4f5c-ad95-70d5d32193ea admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:25.676531 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab853f3-5fc3-4f5c-ad95-70d5d32193ea admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:25.681919 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63c9a0d-7c1c-4cb8-ab81-1c85ca4954b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:25.682549 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63c9a0d-7c1c-4cb8-ab81-1c85ca4954b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:25.682835 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63c9a0d-7c1c-4cb8-ab81-1c85ca4954b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:25.683111 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63c9a0d-7c1c-4cb8-ab81-1c85ca4954b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:25.683820 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63c9a0d-7c1c-4cb8-ab81-1c85ca4954b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:25.685441 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63c9a0d-7c1c-4cb8-ab81-1c85ca4954b9 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:25.695291 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 804/1583] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:16:25 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:16:25.695996 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63c9a0d-7c1c-4cb8-ab81-1c85ca4954b9 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:25.705185 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63c9a0d-7c1c-4cb8-ab81-1c85ca4954b9 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:25.709776 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9ca69c-3615-4c63-9e91-c118ca08267a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:25.710569 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9ca69c-3615-4c63-9e91-c118ca08267a admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:25.710855 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9ca69c-3615-4c63-9e91-c118ca08267a admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:25.711130 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9ca69c-3615-4c63-9e91-c118ca08267a admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:25.711823 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9ca69c-3615-4c63-9e91-c118ca08267a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:25.713356 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9ca69c-3615-4c63-9e91-c118ca08267a admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:25.723446 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 805/1584] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:16:25 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:16:25.724576 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b14b59e-23a3-46dc-99a5-0aacb204c367 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:25.724855 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b14b59e-23a3-46dc-99a5-0aacb204c367 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:25.725135 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b14b59e-23a3-46dc-99a5-0aacb204c367 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:25.736868 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 780/1585] 10.222.0.23 () {60 vars in 1143 bytes} [Sat Jun 18 05:16:25 2022] POST /identity/v3/users => generated 289 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:16:25.736868 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f1c532-cd34-4479-b91d-86f82f7ff0ec admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:25.747874 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f1c532-cd34-4479-b91d-86f82f7ff0ec admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:25.751724 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-1b14b59e-23a3-46dc-99a5-0aacb204c367 None None] MFA Rules not processed for user `4568d73d43804b6c8119b89deece4e99`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:16:25.752661 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690f88a0-5671-4150-bb50-6fddf710987e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:25.752954 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-1b14b59e-23a3-46dc-99a5-0aacb204c367 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:16:25.753569 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690f88a0-5671-4150-bb50-6fddf710987e admin admin] REQUEST_METHOD: `PUT` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:25.753859 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690f88a0-5671-4150-bb50-6fddf710987e admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:25.754138 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690f88a0-5671-4150-bb50-6fddf710987e admin admin] PATH_INFO: `/v3/projects/0b73d42a085a45bfb3ccced90b190a5b/users/79f8817a70b844d7ac49543d74d0e38b/roles/cbf5c4c26898413d93e73b46b41046a3` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:25.766136 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:16:25.769045 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690f88a0-5671-4150-bb50-6fddf710987e admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b73d42a085a45bfb3ccced90b190a5b, user_id=79f8817a70b844d7ac49543d74d0e38b, role_id=cbf5c4c26898413d93e73b46b41046a3)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:25.771766 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690f88a0-5671-4150-bb50-6fddf710987e admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:25.819422 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 806/1586] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:16:25 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:16:25.820235 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b14b59e-23a3-46dc-99a5-0aacb204c367 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:25.830100 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b14b59e-23a3-46dc-99a5-0aacb204c367 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:25.834792 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74167e75-25b5-447a-b5bd-8f5d62183cda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:25.835426 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74167e75-25b5-447a-b5bd-8f5d62183cda admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:25.835712 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74167e75-25b5-447a-b5bd-8f5d62183cda admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:25.835984 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74167e75-25b5-447a-b5bd-8f5d62183cda admin admin] PATH_INFO: `/v3/users` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:25.836660 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74167e75-25b5-447a-b5bd-8f5d62183cda admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:25.838645 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74167e75-25b5-447a-b5bd-8f5d62183cda admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:25.864091 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 781/1587] 10.222.0.23 () {60 vars in 1482 bytes} [Sat Jun 18 05:16:25 2022] PUT /identity/v3/projects/0b73d42a085a45bfb3ccced90b190a5b/users/79f8817a70b844d7ac49543d74d0e38b/roles/cbf5c4c26898413d93e73b46b41046a3 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:16:25.865053 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690f88a0-5671-4150-bb50-6fddf710987e admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:25.914447 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:16:25.931408 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690f88a0-5671-4150-bb50-6fddf710987e admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:25.936184 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0ca655-a599-4e4d-a41a-e13959cabe53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:25.936843 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0ca655-a599-4e4d-a41a-e13959cabe53 admin admin] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:25.937130 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0ca655-a599-4e4d-a41a-e13959cabe53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:25.937431 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0ca655-a599-4e4d-a41a-e13959cabe53 admin admin] PATH_INFO: `/v3/users` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:25.938127 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0ca655-a599-4e4d-a41a-e13959cabe53 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:25.939833 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 807/1588] 10.222.0.23 () {60 vars in 1143 bytes} [Sat Jun 18 05:16:25 2022] POST /identity/v3/users => generated 292 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:16:25.942396 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0ca655-a599-4e4d-a41a-e13959cabe53 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:25.942489 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74167e75-25b5-447a-b5bd-8f5d62183cda admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:25.953087 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74167e75-25b5-447a-b5bd-8f5d62183cda admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:25.957958 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccbbf41-0573-4b3b-8921-72d2539166eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:25.958613 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccbbf41-0573-4b3b-8921-72d2539166eb admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:25.958896 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccbbf41-0573-4b3b-8921-72d2539166eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:25.959173 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccbbf41-0573-4b3b-8921-72d2539166eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:25.960539 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccbbf41-0573-4b3b-8921-72d2539166eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:25.962483 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccbbf41-0573-4b3b-8921-72d2539166eb admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:25.972504 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 808/1589] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:16:25 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:16:25.973300 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccbbf41-0573-4b3b-8921-72d2539166eb admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:25.983934 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccbbf41-0573-4b3b-8921-72d2539166eb admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:25.988736 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f5452c-2512-41a5-89db-dfa534749845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:25.989372 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f5452c-2512-41a5-89db-dfa534749845 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:25.989693 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f5452c-2512-41a5-89db-dfa534749845 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:25.990008 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f5452c-2512-41a5-89db-dfa534749845 admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:25.990732 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f5452c-2512-41a5-89db-dfa534749845 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:25.992415 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f5452c-2512-41a5-89db-dfa534749845 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:26.002951 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 809/1590] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:16:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:16:26.003696 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f5452c-2512-41a5-89db-dfa534749845 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:26.013314 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f5452c-2512-41a5-89db-dfa534749845 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:26.018087 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1e2c6f-3985-4e52-b8d1-0bd2a7949789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:26.024887 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1e2c6f-3985-4e52-b8d1-0bd2a7949789 admin admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:26.024887 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1e2c6f-3985-4e52-b8d1-0bd2a7949789 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:26.024887 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1e2c6f-3985-4e52-b8d1-0bd2a7949789 admin admin] PATH_INFO: `/v3/projects/780f29e257644af280e61247d02a3c1e/users/7cad75680cbb451fad629ee0e0e3d536/roles/cbf5c4c26898413d93e73b46b41046a3` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:26.025161 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 782/1591] 10.222.0.23 () {60 vars in 1143 bytes} [Sat Jun 18 05:16:25 2022] POST /identity/v3/users => generated 279 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:16:26.025161 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0ca655-a599-4e4d-a41a-e13959cabe53 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:26.031621 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0ca655-a599-4e4d-a41a-e13959cabe53 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:26.035599 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1e2c6f-3985-4e52-b8d1-0bd2a7949789 admin admin] RBAC: Authorizing `identity:create_grant(project_id=780f29e257644af280e61247d02a3c1e, user_id=7cad75680cbb451fad629ee0e0e3d536, role_id=cbf5c4c26898413d93e73b46b41046a3)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:26.036327 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b9dcd2-0840-438d-b827-e3266303c225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:26.036979 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b9dcd2-0840-438d-b827-e3266303c225 admin admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:26.037261 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b9dcd2-0840-438d-b827-e3266303c225 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:26.037574 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b9dcd2-0840-438d-b827-e3266303c225 admin admin] PATH_INFO: `/v3/roles` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:26.038884 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1e2c6f-3985-4e52-b8d1-0bd2a7949789 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:26.039141 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b9dcd2-0840-438d-b827-e3266303c225 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:26.040770 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b9dcd2-0840-438d-b827-e3266303c225 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:26.050898 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 783/1592] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:16:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:16:26.051653 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b9dcd2-0840-438d-b827-e3266303c225 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:26.061193 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b9dcd2-0840-438d-b827-e3266303c225 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:26.066035 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8143d341-a184-4dc4-9a4d-b2bb83cb48ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:26.066678 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8143d341-a184-4dc4-9a4d-b2bb83cb48ba admin admin] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:26.066967 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8143d341-a184-4dc4-9a4d-b2bb83cb48ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:26.067248 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8143d341-a184-4dc4-9a4d-b2bb83cb48ba admin admin] PATH_INFO: `/v3/projects` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:26.067929 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8143d341-a184-4dc4-9a4d-b2bb83cb48ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:26.070071 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8143d341-a184-4dc4-9a4d-b2bb83cb48ba admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:26.106255 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 810/1593] 10.222.0.23 () {60 vars in 1482 bytes} [Sat Jun 18 05:16:26 2022] PUT /identity/v3/projects/780f29e257644af280e61247d02a3c1e/users/7cad75680cbb451fad629ee0e0e3d536/roles/cbf5c4c26898413d93e73b46b41046a3 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:16:26.107698 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c073b0-819a-450d-804f-1aa08e4624df None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:26.107989 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c073b0-819a-450d-804f-1aa08e4624df None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:26.108267 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c073b0-819a-450d-804f-1aa08e4624df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:26.125465 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-25c073b0-819a-450d-804f-1aa08e4624df None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:16:26.129443 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-25c073b0-819a-450d-804f-1aa08e4624df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:16:26.183902 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 784/1594] 10.222.0.23 () {60 vars in 1152 bytes} [Sat Jun 18 05:16:26 2022] POST /identity/v3/projects => generated 405 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:16:26.183902 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8143d341-a184-4dc4-9a4d-b2bb83cb48ba admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:26.186596 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 811/1595] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:16:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:16:26.186596 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c073b0-819a-450d-804f-1aa08e4624df None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:26.197919 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c073b0-819a-450d-804f-1aa08e4624df None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:26.247533 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:16:26.247718 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8143d341-a184-4dc4-9a4d-b2bb83cb48ba admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:26.252353 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7f48f4-7b54-46bc-8ef5-d4e92c6a6d67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:26.253011 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7f48f4-7b54-46bc-8ef5-d4e92c6a6d67 admin admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:26.253336 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7f48f4-7b54-46bc-8ef5-d4e92c6a6d67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:26.259934 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7f48f4-7b54-46bc-8ef5-d4e92c6a6d67 admin admin] PATH_INFO: `/v3/roles` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:26.259934 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7f48f4-7b54-46bc-8ef5-d4e92c6a6d67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:26.259934 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7f48f4-7b54-46bc-8ef5-d4e92c6a6d67 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:26.260239 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6942e1e7-b860-440b-baf1-bec4bbf34b28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:26.260239 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6942e1e7-b860-440b-baf1-bec4bbf34b28 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:26.260239 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6942e1e7-b860-440b-baf1-bec4bbf34b28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:26.260239 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6942e1e7-b860-440b-baf1-bec4bbf34b28 admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:26.260599 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6942e1e7-b860-440b-baf1-bec4bbf34b28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:26.260599 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6942e1e7-b860-440b-baf1-bec4bbf34b28 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:26.266971 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 785/1596] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:16:26 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:16:26.267821 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7f48f4-7b54-46bc-8ef5-d4e92c6a6d67 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:26.276336 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 812/1597] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:16:26 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:16:26.276336 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6942e1e7-b860-440b-baf1-bec4bbf34b28 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:26.279719 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7f48f4-7b54-46bc-8ef5-d4e92c6a6d67 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:26.284716 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638c6d04-a4b1-4edd-85d3-c6aa95298ce3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:26.285622 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638c6d04-a4b1-4edd-85d3-c6aa95298ce3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:26.285907 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638c6d04-a4b1-4edd-85d3-c6aa95298ce3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:26.286015 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6942e1e7-b860-440b-baf1-bec4bbf34b28 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:26.286180 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638c6d04-a4b1-4edd-85d3-c6aa95298ce3 admin admin] PATH_INFO: `/v3/projects/d9b8a8dba3d042d797bfb6bf3cef552d/users/f17d7d82e185481da1ddb8e688b2cffe/roles/cbf5c4c26898413d93e73b46b41046a3` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:26.291589 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aae315f-e47e-4bef-8df5-87b2130b41a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:26.292313 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aae315f-e47e-4bef-8df5-87b2130b41a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:26.292599 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aae315f-e47e-4bef-8df5-87b2130b41a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:26.292873 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aae315f-e47e-4bef-8df5-87b2130b41a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:26.293619 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aae315f-e47e-4bef-8df5-87b2130b41a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:26.295221 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aae315f-e47e-4bef-8df5-87b2130b41a9 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:26.301619 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638c6d04-a4b1-4edd-85d3-c6aa95298ce3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9b8a8dba3d042d797bfb6bf3cef552d, user_id=f17d7d82e185481da1ddb8e688b2cffe, role_id=cbf5c4c26898413d93e73b46b41046a3)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:26.305062 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638c6d04-a4b1-4edd-85d3-c6aa95298ce3 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:26.306270 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 813/1598] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:16:26 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:16:26.307584 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aae315f-e47e-4bef-8df5-87b2130b41a9 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:26.317887 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aae315f-e47e-4bef-8df5-87b2130b41a9 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:26.322735 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b788dd8b-a7aa-4d2e-9fd9-8a8df14b50eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:26.323371 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b788dd8b-a7aa-4d2e-9fd9-8a8df14b50eb admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:26.323655 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b788dd8b-a7aa-4d2e-9fd9-8a8df14b50eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:26.323930 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b788dd8b-a7aa-4d2e-9fd9-8a8df14b50eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:26.324644 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b788dd8b-a7aa-4d2e-9fd9-8a8df14b50eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:26.326249 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b788dd8b-a7aa-4d2e-9fd9-8a8df14b50eb admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:26.337607 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 814/1599] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:16:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:16:26.338838 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f7afff-0759-449c-ae9c-c462b2b48764 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:26.339129 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f7afff-0759-449c-ae9c-c462b2b48764 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:26.339401 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f7afff-0759-449c-ae9c-c462b2b48764 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:26.355914 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-09f7afff-0759-449c-ae9c-c462b2b48764 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:16:26.357113 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-09f7afff-0759-449c-ae9c-c462b2b48764 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:16:26.362787 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 786/1600] 10.222.0.23 () {60 vars in 1482 bytes} [Sat Jun 18 05:16:26 2022] PUT /identity/v3/projects/d9b8a8dba3d042d797bfb6bf3cef552d/users/f17d7d82e185481da1ddb8e688b2cffe/roles/cbf5c4c26898413d93e73b46b41046a3 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:16:26.363618 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638c6d04-a4b1-4edd-85d3-c6aa95298ce3 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:26.423030 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 815/1601] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:16:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:16:26.424167 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f7afff-0759-449c-ae9c-c462b2b48764 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:26.440548 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638c6d04-a4b1-4edd-85d3-c6aa95298ce3 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:26.445547 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a67853-c6ef-48f2-afbf-6002e309bcbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:26.446211 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a67853-c6ef-48f2-afbf-6002e309bcbf admin admin] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:26.446497 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a67853-c6ef-48f2-afbf-6002e309bcbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:26.446775 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a67853-c6ef-48f2-afbf-6002e309bcbf admin admin] PATH_INFO: `/v3/users` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:26.447460 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a67853-c6ef-48f2-afbf-6002e309bcbf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:26.449376 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a67853-c6ef-48f2-afbf-6002e309bcbf admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:26.478713 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f7afff-0759-449c-ae9c-c462b2b48764 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:26.483394 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454b5bc5-fd93-4cce-828e-72679a4910b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c4d7cde8f4ad9aa479bb09e325db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:26.484033 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454b5bc5-fd93-4cce-828e-72679a4910b1 service neutron] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:26.484316 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454b5bc5-fd93-4cce-828e-72679a4910b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:26.484592 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454b5bc5-fd93-4cce-828e-72679a4910b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:26.494272 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454b5bc5-fd93-4cce-828e-72679a4910b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:26.497030 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454b5bc5-fd93-4cce-828e-72679a4910b1 service neutron] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:26.505284 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 816/1602] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:16:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:16:26.506046 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454b5bc5-fd93-4cce-828e-72679a4910b1 service neutron] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:26.515919 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454b5bc5-fd93-4cce-828e-72679a4910b1 service neutron] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:26.520883 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb7d6c0-88fd-45ff-8bd8-6d50ea686fc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:26.521594 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb7d6c0-88fd-45ff-8bd8-6d50ea686fc8 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:26.527178 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb7d6c0-88fd-45ff-8bd8-6d50ea686fc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:26.527178 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb7d6c0-88fd-45ff-8bd8-6d50ea686fc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:26.527178 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb7d6c0-88fd-45ff-8bd8-6d50ea686fc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:26.527178 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb7d6c0-88fd-45ff-8bd8-6d50ea686fc8 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:26.535188 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 817/1603] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:16:26 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:16:26.535947 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb7d6c0-88fd-45ff-8bd8-6d50ea686fc8 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:26.545725 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:16:26.547548 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb7d6c0-88fd-45ff-8bd8-6d50ea686fc8 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:26.549013 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 787/1604] 10.222.0.23 () {60 vars in 1143 bytes} [Sat Jun 18 05:16:26 2022] POST /identity/v3/users => generated 309 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:16:26.549904 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a67853-c6ef-48f2-afbf-6002e309bcbf admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:26.555811 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0b20ba-9903-422d-b46e-3d3c73368ed8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:26.555811 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0b20ba-9903-422d-b46e-3d3c73368ed8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:26.555811 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0b20ba-9903-422d-b46e-3d3c73368ed8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:26.555811 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0b20ba-9903-422d-b46e-3d3c73368ed8 admin admin] PATH_INFO: `/v3/projects/758883a5aea04c85ad65bb6970bdb098/users/e0f3999a062e45748878a3204e3ea456/roles/cbf5c4c26898413d93e73b46b41046a3` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:26.562256 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a67853-c6ef-48f2-afbf-6002e309bcbf admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:26.567361 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e5b10e-85e3-425e-8b99-761d7a28fe88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:26.568107 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e5b10e-85e3-425e-8b99-761d7a28fe88 admin admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:26.568404 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e5b10e-85e3-425e-8b99-761d7a28fe88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:26.568690 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e5b10e-85e3-425e-8b99-761d7a28fe88 admin admin] PATH_INFO: `/v3/roles` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:26.569485 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e5b10e-85e3-425e-8b99-761d7a28fe88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:26.571247 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e5b10e-85e3-425e-8b99-761d7a28fe88 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:26.571788 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0b20ba-9903-422d-b46e-3d3c73368ed8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=758883a5aea04c85ad65bb6970bdb098, user_id=e0f3999a062e45748878a3204e3ea456, role_id=cbf5c4c26898413d93e73b46b41046a3)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:26.574453 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0b20ba-9903-422d-b46e-3d3c73368ed8 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:26.581607 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 788/1605] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:16:26 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:16:26.582387 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e5b10e-85e3-425e-8b99-761d7a28fe88 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:26.592647 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e5b10e-85e3-425e-8b99-761d7a28fe88 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:26.597439 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66daaf4e-c1f0-403b-b318-804ccbc89e70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:26.601800 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66daaf4e-c1f0-403b-b318-804ccbc89e70 admin admin] REQUEST_METHOD: `PUT` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:26.601800 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66daaf4e-c1f0-403b-b318-804ccbc89e70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:26.601800 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66daaf4e-c1f0-403b-b318-804ccbc89e70 admin admin] PATH_INFO: `/v3/projects/969c470f720c4b2f97ba9f45c5d0d426/users/d2c8f964ee6a49cda501ed28a9432c8e/roles/cbf5c4c26898413d93e73b46b41046a3` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:26.617199 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66daaf4e-c1f0-403b-b318-804ccbc89e70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=969c470f720c4b2f97ba9f45c5d0d426, user_id=d2c8f964ee6a49cda501ed28a9432c8e, role_id=cbf5c4c26898413d93e73b46b41046a3)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:26.624811 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66daaf4e-c1f0-403b-b318-804ccbc89e70 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:26.675457 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 818/1606] 10.222.0.23 () {60 vars in 1482 bytes} [Sat Jun 18 05:16:26 2022] PUT /identity/v3/projects/758883a5aea04c85ad65bb6970bdb098/users/e0f3999a062e45748878a3204e3ea456/roles/cbf5c4c26898413d93e73b46b41046a3 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:16:26.676979 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1df55ac-c589-4557-9464-e49856d109e2 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:26.677461 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1df55ac-c589-4557-9464-e49856d109e2 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:26.677884 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1df55ac-c589-4557-9464-e49856d109e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:26.704110 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 789/1607] 10.222.0.23 () {60 vars in 1482 bytes} [Sat Jun 18 05:16:26 2022] PUT /identity/v3/projects/969c470f720c4b2f97ba9f45c5d0d426/users/d2c8f964ee6a49cda501ed28a9432c8e/roles/cbf5c4c26898413d93e73b46b41046a3 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:16:26.709462 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66daaf4e-c1f0-403b-b318-804ccbc89e70 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:26.710088 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-b1df55ac-c589-4557-9464-e49856d109e2 None None] MFA Rules not processed for user `f17d7d82e185481da1ddb8e688b2cffe`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:16:26.710732 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-b1df55ac-c589-4557-9464-e49856d109e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:16:26.735689 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:16:26.762476 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66daaf4e-c1f0-403b-b318-804ccbc89e70 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:26.766398 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aeaeec4-c287-481b-8f66-b4a7a2d1560e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c4d7cde8f4ad9aa479bb09e325db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:26.767047 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeaeec4-c287-481b-8f66-b4a7a2d1560e service neutron] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:26.767333 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeaeec4-c287-481b-8f66-b4a7a2d1560e service neutron] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:26.767612 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeaeec4-c287-481b-8f66-b4a7a2d1560e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:26.776891 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 819/1608] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:16:26 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:16:26.781268 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1df55ac-c589-4557-9464-e49856d109e2 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:26.853054 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aeaeec4-c287-481b-8f66-b4a7a2d1560e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:26.854687 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aeaeec4-c287-481b-8f66-b4a7a2d1560e service neutron] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:26.865077 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1df55ac-c589-4557-9464-e49856d109e2 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:26.868573 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 790/1609] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:16:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:16:26.876428 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aeaeec4-c287-481b-8f66-b4a7a2d1560e service neutron] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:26.876576 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a79410-c3aa-48a2-a5cc-a8a6769a2ca8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:26.876576 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a79410-c3aa-48a2-a5cc-a8a6769a2ca8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:26.876576 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a79410-c3aa-48a2-a5cc-a8a6769a2ca8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:26.876576 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a79410-c3aa-48a2-a5cc-a8a6769a2ca8 admin admin] PATH_INFO: `/v3/projects/b97df413b0b140a0aceac04617fb59bd/users/ef7fdbe955364009a891d2b6cbef1db1/roles/cbf5c4c26898413d93e73b46b41046a3` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:26.880543 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aeaeec4-c287-481b-8f66-b4a7a2d1560e service neutron] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:26.886665 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d7c957-6885-4715-8086-0546021a6b6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:26.887337 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d7c957-6885-4715-8086-0546021a6b6b admin admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:26.887811 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d7c957-6885-4715-8086-0546021a6b6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:26.887927 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d7c957-6885-4715-8086-0546021a6b6b admin admin] PATH_INFO: `/v3/roles` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:26.888634 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d7c957-6885-4715-8086-0546021a6b6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:26.890254 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d7c957-6885-4715-8086-0546021a6b6b admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:26.895327 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a79410-c3aa-48a2-a5cc-a8a6769a2ca8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b97df413b0b140a0aceac04617fb59bd, user_id=ef7fdbe955364009a891d2b6cbef1db1, role_id=cbf5c4c26898413d93e73b46b41046a3)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:26.899851 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a79410-c3aa-48a2-a5cc-a8a6769a2ca8 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:26.902089 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 791/1610] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:16:26 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:16:26.902932 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d7c957-6885-4715-8086-0546021a6b6b admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:26.913340 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d7c957-6885-4715-8086-0546021a6b6b admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:26.918510 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bda4c6-7bb9-4f7b-869a-35d38fe2eab7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:26.919390 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bda4c6-7bb9-4f7b-869a-35d38fe2eab7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:26.919673 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bda4c6-7bb9-4f7b-869a-35d38fe2eab7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:26.919949 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bda4c6-7bb9-4f7b-869a-35d38fe2eab7 admin admin] PATH_INFO: `/v3/projects/c8dc6016d2404e0e8fb4f303fa12427d/users/f90bd977ff8f4406bbcf63c4853bbf68/roles/cbf5c4c26898413d93e73b46b41046a3` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:26.936338 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bda4c6-7bb9-4f7b-869a-35d38fe2eab7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8dc6016d2404e0e8fb4f303fa12427d, user_id=f90bd977ff8f4406bbcf63c4853bbf68, role_id=cbf5c4c26898413d93e73b46b41046a3)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:26.939099 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bda4c6-7bb9-4f7b-869a-35d38fe2eab7 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:26.997355 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 820/1611] 10.222.0.23 () {60 vars in 1482 bytes} [Sat Jun 18 05:16:26 2022] PUT /identity/v3/projects/b97df413b0b140a0aceac04617fb59bd/users/ef7fdbe955364009a891d2b6cbef1db1/roles/cbf5c4c26898413d93e73b46b41046a3 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:16:26.998864 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9116e98-c32e-4754-84e1-f46ec40a093e None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:26.999156 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9116e98-c32e-4754-84e1-f46ec40a093e None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:26.999439 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9116e98-c32e-4754-84e1-f46ec40a093e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:27.019752 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-e9116e98-c32e-4754-84e1-f46ec40a093e None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:16:27.019752 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-e9116e98-c32e-4754-84e1-f46ec40a093e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:16:27.065763 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 792/1612] 10.222.0.23 () {60 vars in 1482 bytes} [Sat Jun 18 05:16:26 2022] PUT /identity/v3/projects/c8dc6016d2404e0e8fb4f303fa12427d/users/f90bd977ff8f4406bbcf63c4853bbf68/roles/cbf5c4c26898413d93e73b46b41046a3 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:16:27.067164 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4351b6f-570d-45d8-b3f1-6c9e715074bc None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:27.067456 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4351b6f-570d-45d8-b3f1-6c9e715074bc None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:27.067737 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4351b6f-570d-45d8-b3f1-6c9e715074bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:27.087460 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-c4351b6f-570d-45d8-b3f1-6c9e715074bc None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:16:27.092047 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-c4351b6f-570d-45d8-b3f1-6c9e715074bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:16:27.092172 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 821/1613] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:16:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:16:27.092172 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9116e98-c32e-4754-84e1-f46ec40a093e None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:27.132557 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:16:27.159604 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 793/1614] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:16:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:16:27.160499 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4351b6f-570d-45d8-b3f1-6c9e715074bc None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:27.164387 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:16:27.180154 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4351b6f-570d-45d8-b3f1-6c9e715074bc None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:27.180268 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9116e98-c32e-4754-84e1-f46ec40a093e None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:27.180370 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b517be-4c48-4dc7-a503-fdd53d565a69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:27.180370 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b517be-4c48-4dc7-a503-fdd53d565a69 admin admin] REQUEST_METHOD: `PUT` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:27.180370 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b517be-4c48-4dc7-a503-fdd53d565a69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:27.180658 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b517be-4c48-4dc7-a503-fdd53d565a69 admin admin] PATH_INFO: `/v3/projects/0611875730fb403c95901c0c9de52aa5/users/59c8f79e06ec4d32a0c7fe54308810f1/roles/21c65f2c51bd4dd283d13507d2714467` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:27.199283 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b517be-4c48-4dc7-a503-fdd53d565a69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0611875730fb403c95901c0c9de52aa5, user_id=59c8f79e06ec4d32a0c7fe54308810f1, role_id=21c65f2c51bd4dd283d13507d2714467)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:27.202915 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b517be-4c48-4dc7-a503-fdd53d565a69 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:27.262624 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c484e5-aca0-467c-b37c-13e708258f9c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9de0428abc8400aa338aba4f7c34c94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:27.262624 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c484e5-aca0-467c-b37c-13e708258f9c service cinder] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:27.262624 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c484e5-aca0-467c-b37c-13e708258f9c service cinder] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:27.262624 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c484e5-aca0-467c-b37c-13e708258f9c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:27.316019 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 794/1615] 10.222.0.23 () {60 vars in 1482 bytes} [Sat Jun 18 05:16:27 2022] PUT /identity/v3/projects/0611875730fb403c95901c0c9de52aa5/users/59c8f79e06ec4d32a0c7fe54308810f1/roles/21c65f2c51bd4dd283d13507d2714467 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:16:27.318030 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c484e5-aca0-467c-b37c-13e708258f9c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:27.321490 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c484e5-aca0-467c-b37c-13e708258f9c service cinder] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:27.321616 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdb040e-661c-4364-8eb0-dcd12a35dbc6 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:27.321616 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdb040e-661c-4364-8eb0-dcd12a35dbc6 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:27.321616 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdb040e-661c-4364-8eb0-dcd12a35dbc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:27.329235 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 822/1616] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:16:27 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:16:27.330553 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dbc158-73b6-489a-9e43-7002f90cd6d5 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:27.330840 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dbc158-73b6-489a-9e43-7002f90cd6d5 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:27.331118 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dbc158-73b6-489a-9e43-7002f90cd6d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:27.370239 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-dcdb040e-661c-4364-8eb0-dcd12a35dbc6 None None] MFA Rules not processed for user `ef7fdbe955364009a891d2b6cbef1db1`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:16:27.371480 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-dcdb040e-661c-4364-8eb0-dcd12a35dbc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:16:27.391733 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-52dbc158-73b6-489a-9e43-7002f90cd6d5 None None] MFA Rules not processed for user `f90bd977ff8f4406bbcf63c4853bbf68`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:16:27.394786 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-52dbc158-73b6-489a-9e43-7002f90cd6d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:16:27.515296 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 795/1617] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:16:27 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:16:27.516445 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdb040e-661c-4364-8eb0-dcd12a35dbc6 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:27.518166 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-dcdb040e-661c-4364-8eb0-dcd12a35dbc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:16:27.553105 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 823/1618] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:16:27 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:16:27.557119 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dbc158-73b6-489a-9e43-7002f90cd6d5 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:27.561029 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-52dbc158-73b6-489a-9e43-7002f90cd6d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:16:27.600716 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdb040e-661c-4364-8eb0-dcd12a35dbc6 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:27.605023 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62354152-6d2d-4eaa-9629-709b7f20e186 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c4d7cde8f4ad9aa479bb09e325db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:27.605729 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62354152-6d2d-4eaa-9629-709b7f20e186 service neutron] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:27.606014 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62354152-6d2d-4eaa-9629-709b7f20e186 service neutron] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:27.606293 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62354152-6d2d-4eaa-9629-709b7f20e186 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:27.641053 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:16:27.645156 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dbc158-73b6-489a-9e43-7002f90cd6d5 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:27.650645 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50108388-3744-43e0-a665-0ae5007666e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c4d7cde8f4ad9aa479bb09e325db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:27.651393 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50108388-3744-43e0-a665-0ae5007666e7 service neutron] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:27.651814 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50108388-3744-43e0-a665-0ae5007666e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:27.652205 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50108388-3744-43e0-a665-0ae5007666e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:27.694572 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:16:27.696862 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62354152-6d2d-4eaa-9629-709b7f20e186 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:27.698601 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62354152-6d2d-4eaa-9629-709b7f20e186 service neutron] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:27.710414 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 796/1619] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:16:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:16:27.711324 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62354152-6d2d-4eaa-9629-709b7f20e186 service neutron] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:27.721327 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62354152-6d2d-4eaa-9629-709b7f20e186 service neutron] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:27.731762 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50108388-3744-43e0-a665-0ae5007666e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:27.733317 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50108388-3744-43e0-a665-0ae5007666e7 service neutron] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:27.734281 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68aa8ebf-eafa-46f9-9b09-9e161ff2f461 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:27.735029 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68aa8ebf-eafa-46f9-9b09-9e161ff2f461 admin admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:27.735384 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68aa8ebf-eafa-46f9-9b09-9e161ff2f461 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:27.735724 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68aa8ebf-eafa-46f9-9b09-9e161ff2f461 admin admin] PATH_INFO: `/v3/roles` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:27.736538 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68aa8ebf-eafa-46f9-9b09-9e161ff2f461 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:27.738298 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68aa8ebf-eafa-46f9-9b09-9e161ff2f461 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:27.750084 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 797/1620] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:16:27 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:16:27.751462 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b66eb75-df09-4f4b-a3f0-f0ff6da57db1 None None] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:27.755282 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b66eb75-df09-4f4b-a3f0-f0ff6da57db1 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:27.755282 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b66eb75-df09-4f4b-a3f0-f0ff6da57db1 None None] PATH_INFO: `/` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:27.755526 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 824/1621] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:16:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:16:27.755650 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 798/1622] 10.222.0.23 () {54 vars in 961 bytes} [Sat Jun 18 05:16:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 05:16:27.755650 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b66eb75-df09-4f4b-a3f0-f0ff6da57db1 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:27.761966 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50108388-3744-43e0-a665-0ae5007666e7 service neutron] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:27.769014 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b66eb75-df09-4f4b-a3f0-f0ff6da57db1 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:27.774870 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd2a45b-944d-4ab0-95c6-4ad033b55bfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:27.775037 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50108388-3744-43e0-a665-0ae5007666e7 service neutron] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:27.775950 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd2a45b-944d-4ab0-95c6-4ad033b55bfd admin admin] REQUEST_METHOD: `PUT` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:27.776304 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd2a45b-944d-4ab0-95c6-4ad033b55bfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:27.776656 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd2a45b-944d-4ab0-95c6-4ad033b55bfd admin admin] PATH_INFO: `/v3/projects/0611875730fb403c95901c0c9de52aa5/users/59c8f79e06ec4d32a0c7fe54308810f1/roles/21c65f2c51bd4dd283d13507d2714467` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:27.782713 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425ebb75-b200-4874-80f8-2b8efef644aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:27.784006 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd2a45b-944d-4ab0-95c6-4ad033b55bfd admin admin] RBAC: Authorizing `identity:create_grant(project_id=0611875730fb403c95901c0c9de52aa5, user_id=59c8f79e06ec4d32a0c7fe54308810f1, role_id=21c65f2c51bd4dd283d13507d2714467)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:27.784386 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425ebb75-b200-4874-80f8-2b8efef644aa admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:27.784774 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425ebb75-b200-4874-80f8-2b8efef644aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:27.785148 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425ebb75-b200-4874-80f8-2b8efef644aa admin admin] PATH_INFO: `/v3/projects` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:27.793018 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425ebb75-b200-4874-80f8-2b8efef644aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:27.793018 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425ebb75-b200-4874-80f8-2b8efef644aa admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:27.797705 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd2a45b-944d-4ab0-95c6-4ad033b55bfd admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:27.825069 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 799/1623] 10.222.0.23 () {60 vars in 1482 bytes} [Sat Jun 18 05:16:27 2022] PUT /identity/v3/projects/0611875730fb403c95901c0c9de52aa5/users/59c8f79e06ec4d32a0c7fe54308810f1/roles/21c65f2c51bd4dd283d13507d2714467 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:16:27.825069 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f8a7fc-500c-4bd5-ab40-28772b7b194e None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:27.825345 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f8a7fc-500c-4bd5-ab40-28772b7b194e None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:27.825751 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f8a7fc-500c-4bd5-ab40-28772b7b194e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:27.850969 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-a7f8a7fc-500c-4bd5-ab40-28772b7b194e None None] MFA Rules not processed for user `15e7f45ee0744c1d9602187eac8e8b80`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:16:27.852314 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-a7f8a7fc-500c-4bd5-ab40-28772b7b194e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:16:27.878774 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 825/1624] 10.222.0.23 () {60 vars in 1152 bytes} [Sat Jun 18 05:16:27 2022] POST /identity/v3/projects => generated 357 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:16:27.880542 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8444cd8f-f45a-46fe-ab91-bda8eb7172dd None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:27.880908 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8444cd8f-f45a-46fe-ab91-bda8eb7172dd None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:27.881279 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8444cd8f-f45a-46fe-ab91-bda8eb7172dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:27.914855 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-8444cd8f-f45a-46fe-ab91-bda8eb7172dd None None] MFA Rules not processed for user `59c8f79e06ec4d32a0c7fe54308810f1`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:16:27.916230 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-8444cd8f-f45a-46fe-ab91-bda8eb7172dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:16:27.987287 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 800/1625] 10.222.0.23 () {58 vars in 1058 bytes} [Sat Jun 18 05:16:27 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:16:27.988512 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f8a7fc-500c-4bd5-ab40-28772b7b194e None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:28.024618 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 826/1626] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:16:27 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:16:28.025942 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8444cd8f-f45a-46fe-ab91-bda8eb7172dd None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:28.092208 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f8a7fc-500c-4bd5-ab40-28772b7b194e None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:28.129087 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:16:28.142542 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8444cd8f-f45a-46fe-ab91-bda8eb7172dd None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:28.149152 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ec2fa2-8713-4a64-aad0-5d4bbf7c1aeb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15e7f45ee0744c1d9602187eac8e8b80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:28.149995 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ec2fa2-8713-4a64-aad0-5d4bbf7c1aeb service nova] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:28.150349 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ec2fa2-8713-4a64-aad0-5d4bbf7c1aeb service nova] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:28.150697 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ec2fa2-8713-4a64-aad0-5d4bbf7c1aeb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:28.226102 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c36e9a7-5421-40f7-9593-7b6347699e93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:28.227022 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c36e9a7-5421-40f7-9593-7b6347699e93 admin admin] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:28.228449 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c36e9a7-5421-40f7-9593-7b6347699e93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:28.228742 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c36e9a7-5421-40f7-9593-7b6347699e93 admin admin] PATH_INFO: `/v3/users` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:28.229486 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c36e9a7-5421-40f7-9593-7b6347699e93 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:28.239916 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c36e9a7-5421-40f7-9593-7b6347699e93 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:28.247583 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ec2fa2-8713-4a64-aad0-5d4bbf7c1aeb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:28.250959 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ec2fa2-8713-4a64-aad0-5d4bbf7c1aeb service nova] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:28.260968 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 827/1627] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:16:28 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:16:28.267291 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ec2fa2-8713-4a64-aad0-5d4bbf7c1aeb service nova] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:28.375438 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 801/1628] 10.222.0.23 () {60 vars in 1143 bytes} [Sat Jun 18 05:16:27 2022] POST /identity/v3/users => generated 285 bytes in 387 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:16:28.386536 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c36e9a7-5421-40f7-9593-7b6347699e93 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:28.425918 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c36e9a7-5421-40f7-9593-7b6347699e93 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:28.425918 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520909a8-33f7-4c09-bb96-ff40b8fd645a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:28.425918 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520909a8-33f7-4c09-bb96-ff40b8fd645a admin admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:28.425918 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520909a8-33f7-4c09-bb96-ff40b8fd645a admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:28.426289 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520909a8-33f7-4c09-bb96-ff40b8fd645a admin admin] PATH_INFO: `/v3/roles` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:28.426484 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520909a8-33f7-4c09-bb96-ff40b8fd645a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:28.430573 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520909a8-33f7-4c09-bb96-ff40b8fd645a admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:28.433283 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ec2fa2-8713-4a64-aad0-5d4bbf7c1aeb service nova] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:28.438032 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b3602c-1beb-49c4-abb0-dd053adfc0c1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9de0428abc8400aa338aba4f7c34c94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:28.438694 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b3602c-1beb-49c4-abb0-dd053adfc0c1 service cinder] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:28.438994 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b3602c-1beb-49c4-abb0-dd053adfc0c1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:28.439297 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b3602c-1beb-49c4-abb0-dd053adfc0c1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:28.446696 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 802/1629] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:16:28 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:16:28.453728 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520909a8-33f7-4c09-bb96-ff40b8fd645a admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:28.457371 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b3602c-1beb-49c4-abb0-dd053adfc0c1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:28.459051 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b3602c-1beb-49c4-abb0-dd053adfc0c1 service cinder] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:28.460992 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520909a8-33f7-4c09-bb96-ff40b8fd645a admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:28.466925 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c1ec71-8563-4870-8ff3-91a2fa279c14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:28.468809 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c1ec71-8563-4870-8ff3-91a2fa279c14 admin admin] REQUEST_METHOD: `PUT` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:28.469163 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c1ec71-8563-4870-8ff3-91a2fa279c14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:28.469586 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c1ec71-8563-4870-8ff3-91a2fa279c14 admin admin] PATH_INFO: `/v3/projects/98533cb841214d4c9560c4e457b41b7e/users/c204971ad01e48b7b071334717286d52/roles/21c65f2c51bd4dd283d13507d2714467` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:28.489033 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 828/1630] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:16:28 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:16:28.502741 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c1ec71-8563-4870-8ff3-91a2fa279c14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=98533cb841214d4c9560c4e457b41b7e, user_id=c204971ad01e48b7b071334717286d52, role_id=21c65f2c51bd4dd283d13507d2714467)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:28.510277 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c1ec71-8563-4870-8ff3-91a2fa279c14 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:28.616028 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 803/1631] 10.222.0.23 () {60 vars in 1482 bytes} [Sat Jun 18 05:16:28 2022] PUT /identity/v3/projects/98533cb841214d4c9560c4e457b41b7e/users/c204971ad01e48b7b071334717286d52/roles/21c65f2c51bd4dd283d13507d2714467 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:16:28.620753 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b3602c-1beb-49c4-abb0-dd053adfc0c1 service cinder] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:28.638989 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b3602c-1beb-49c4-abb0-dd053adfc0c1 service cinder] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:28.727673 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a8b073-a169-4ba8-9568-20cf05ff7dc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:28.728408 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a8b073-a169-4ba8-9568-20cf05ff7dc1 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:28.728695 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a8b073-a169-4ba8-9568-20cf05ff7dc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:28.728966 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6f8018-c696-4423-b618-5e8de1bd3b27 None None] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:28.729101 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a8b073-a169-4ba8-9568-20cf05ff7dc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:28.729279 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6f8018-c696-4423-b618-5e8de1bd3b27 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:28.729546 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6f8018-c696-4423-b618-5e8de1bd3b27 None None] PATH_INFO: `/` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:28.729865 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a8b073-a169-4ba8-9568-20cf05ff7dc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:28.731220 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 804/1632] 10.222.0.23 () {54 vars in 912 bytes} [Sat Jun 18 05:16:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 18 05:16:28.731653 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a8b073-a169-4ba8-9568-20cf05ff7dc1 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:28.737052 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e411d546-1567-46f0-ba41-28b71cbe0652 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:28.737343 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e411d546-1567-46f0-ba41-28b71cbe0652 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:28.737632 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e411d546-1567-46f0-ba41-28b71cbe0652 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:28.749182 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 829/1633] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:16:28 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:16:28.774496 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a8b073-a169-4ba8-9568-20cf05ff7dc1 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:28.791078 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-e411d546-1567-46f0-ba41-28b71cbe0652 None None] MFA Rules not processed for user `15e7f45ee0744c1d9602187eac8e8b80`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:16:28.794831 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-e411d546-1567-46f0-ba41-28b71cbe0652 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:16:28.809257 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a8b073-a169-4ba8-9568-20cf05ff7dc1 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:28.822010 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018d39be-25b5-43ef-8380-012c0d5cc3eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:28.823087 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018d39be-25b5-43ef-8380-012c0d5cc3eb admin admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:28.823375 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018d39be-25b5-43ef-8380-012c0d5cc3eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:28.823652 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018d39be-25b5-43ef-8380-012c0d5cc3eb admin admin] PATH_INFO: `/v3/projects/98533cb841214d4c9560c4e457b41b7e/users/c204971ad01e48b7b071334717286d52/roles/21c65f2c51bd4dd283d13507d2714467` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:28.826098 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018d39be-25b5-43ef-8380-012c0d5cc3eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=98533cb841214d4c9560c4e457b41b7e, user_id=c204971ad01e48b7b071334717286d52, role_id=21c65f2c51bd4dd283d13507d2714467)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:28.829936 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018d39be-25b5-43ef-8380-012c0d5cc3eb admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:28.847100 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 830/1634] 10.222.0.23 () {60 vars in 1482 bytes} [Sat Jun 18 05:16:28 2022] PUT /identity/v3/projects/98533cb841214d4c9560c4e457b41b7e/users/c204971ad01e48b7b071334717286d52/roles/21c65f2c51bd4dd283d13507d2714467 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 18 05:16:28.861997 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:16:28.879285 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c54680-5bbe-47ee-951f-09d4a247afb9 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:28.879526 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c54680-5bbe-47ee-951f-09d4a247afb9 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:28.879779 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c54680-5bbe-47ee-951f-09d4a247afb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:28.945304 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-48c54680-5bbe-47ee-951f-09d4a247afb9 None None] MFA Rules not processed for user `c204971ad01e48b7b071334717286d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:16:28.948970 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-48c54680-5bbe-47ee-951f-09d4a247afb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:16:28.980808 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 805/1635] 10.222.0.23 () {58 vars in 1009 bytes} [Sat Jun 18 05:16:28 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 18 05:16:28.997261 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e411d546-1567-46f0-ba41-28b71cbe0652 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:29.083458 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 831/1636] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:16:28 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:16:29.088972 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c54680-5bbe-47ee-951f-09d4a247afb9 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:29.102573 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e411d546-1567-46f0-ba41-28b71cbe0652 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:29.163536 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:16:29.174112 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea20c8a-1b2e-47e0-a433-3cb11a6abe86 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9de0428abc8400aa338aba4f7c34c94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:29.174920 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea20c8a-1b2e-47e0-a433-3cb11a6abe86 service cinder] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:29.175209 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea20c8a-1b2e-47e0-a433-3cb11a6abe86 service cinder] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:29.175606 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea20c8a-1b2e-47e0-a433-3cb11a6abe86 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:29.197530 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c54680-5bbe-47ee-951f-09d4a247afb9 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:29.202985 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1993b1-f6d2-4cb9-8cdd-3bab77563e6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:29.204006 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1993b1-f6d2-4cb9-8cdd-3bab77563e6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:29.204288 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1993b1-f6d2-4cb9-8cdd-3bab77563e6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:29.204563 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1993b1-f6d2-4cb9-8cdd-3bab77563e6d admin admin] PATH_INFO: `/v3/users/4568d73d43804b6c8119b89deece4e99` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:29.207121 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1993b1-f6d2-4cb9-8cdd-3bab77563e6d admin admin] RBAC: Authorizing `identity:delete_user(user_id=4568d73d43804b6c8119b89deece4e99)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:29.209152 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1993b1-f6d2-4cb9-8cdd-3bab77563e6d admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:29.316750 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea20c8a-1b2e-47e0-a433-3cb11a6abe86 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:29.318305 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea20c8a-1b2e-47e0-a433-3cb11a6abe86 service cinder] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:29.331458 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 806/1637] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:16:28 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:16:29.332489 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea20c8a-1b2e-47e0-a433-3cb11a6abe86 service cinder] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:29.344056 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea20c8a-1b2e-47e0-a433-3cb11a6abe86 service cinder] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:29.347365 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f127a0-33ae-46ad-875e-c962da480a4f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9de0428abc8400aa338aba4f7c34c94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:29.348257 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f127a0-33ae-46ad-875e-c962da480a4f service cinder] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:29.348691 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f127a0-33ae-46ad-875e-c962da480a4f service cinder] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:29.349116 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f127a0-33ae-46ad-875e-c962da480a4f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:29.361845 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f127a0-33ae-46ad-875e-c962da480a4f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:29.364345 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f127a0-33ae-46ad-875e-c962da480a4f service cinder] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:29.396809 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 807/1638] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:16:29 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:16:29.399824 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc110e96-0195-432a-8cf9-4ca49d8e633e None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:29.401061 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc110e96-0195-432a-8cf9-4ca49d8e633e None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:29.401061 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc110e96-0195-432a-8cf9-4ca49d8e633e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:29.441815 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-6f1993b1-f6d2-4cb9-8cdd-3bab77563e6d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4568d73d43804b6c8119b89deece4e99'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:16:29.459075 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-fc110e96-0195-432a-8cf9-4ca49d8e633e None None] MFA Rules not processed for user `7cad75680cbb451fad629ee0e0e3d536`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:16:29.467168 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-fc110e96-0195-432a-8cf9-4ca49d8e633e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:16:29.472809 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-6f1993b1-f6d2-4cb9-8cdd-3bab77563e6d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4568d73d43804b6c8119b89deece4e99'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:16:29.474008 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-6f1993b1-f6d2-4cb9-8cdd-3bab77563e6d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4568d73d43804b6c8119b89deece4e99'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:16:29.568266 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-6f1993b1-f6d2-4cb9-8cdd-3bab77563e6d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4568d73d43804b6c8119b89deece4e99'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:16:29.577292 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-6f1993b1-f6d2-4cb9-8cdd-3bab77563e6d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4568d73d43804b6c8119b89deece4e99'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:16:29.596032 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 832/1639] 10.222.0.23 () {58 vars in 1223 bytes} [Sat Jun 18 05:16:29 2022] DELETE /identity/v3/users/4568d73d43804b6c8119b89deece4e99 => generated 0 bytes in 509 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:16:29.597815 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1b36f8-7871-4a81-859d-b89d359b0637 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:29.598110 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1b36f8-7871-4a81-859d-b89d359b0637 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:29.598530 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1b36f8-7871-4a81-859d-b89d359b0637 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:29.635461 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 808/1640] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:16:29 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:16:29.636379 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-2d1b36f8-7871-4a81-859d-b89d359b0637 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:16:29.636514 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc110e96-0195-432a-8cf9-4ca49d8e633e None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:29.637949 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-2d1b36f8-7871-4a81-859d-b89d359b0637 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:16:29.638355 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-fc110e96-0195-432a-8cf9-4ca49d8e633e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:16:29.684520 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc110e96-0195-432a-8cf9-4ca49d8e633e None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:29.767194 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 833/1641] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:16:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:16:29.783400 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1b36f8-7871-4a81-859d-b89d359b0637 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:29.784885 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-2d1b36f8-7871-4a81-859d-b89d359b0637 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:16:29.795850 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5fdce7-1427-4cff-876c-00d7a1decf78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:29.797642 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5fdce7-1427-4cff-876c-00d7a1decf78 admin admin] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:29.797941 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5fdce7-1427-4cff-876c-00d7a1decf78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:29.799478 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5fdce7-1427-4cff-876c-00d7a1decf78 admin admin] PATH_INFO: `/v3/projects` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:29.801518 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5fdce7-1427-4cff-876c-00d7a1decf78 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:29.809533 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5fdce7-1427-4cff-876c-00d7a1decf78 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:29.894049 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1b36f8-7871-4a81-859d-b89d359b0637 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:29.900321 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0e1027-8241-4f9f-ac14-0e130a274b4a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c4d7cde8f4ad9aa479bb09e325db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:29.901098 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0e1027-8241-4f9f-ac14-0e130a274b4a service neutron] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:29.901562 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0e1027-8241-4f9f-ac14-0e130a274b4a service neutron] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:29.902068 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0e1027-8241-4f9f-ac14-0e130a274b4a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:29.921355 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0e1027-8241-4f9f-ac14-0e130a274b4a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:29.921355 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0e1027-8241-4f9f-ac14-0e130a274b4a service neutron] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:29.934044 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 834/1643] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:16:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 18 05:16:29.934820 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 809/1642] 10.222.0.23 () {60 vars in 1152 bytes} [Sat Jun 18 05:16:29 2022] POST /identity/v3/projects => generated 403 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 18 05:16:29.934957 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0e1027-8241-4f9f-ac14-0e130a274b4a service neutron] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:29.935877 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5fdce7-1427-4cff-876c-00d7a1decf78 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:29.936281 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-6e0e1027-8241-4f9f-ac14-0e130a274b4a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:16:30.024213 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0e1027-8241-4f9f-ac14-0e130a274b4a service neutron] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:30.027315 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:16:30.031848 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5fdce7-1427-4cff-876c-00d7a1decf78 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:30.037634 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c249330-a0e5-4668-a2de-4a412c7bedf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:30.038974 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c249330-a0e5-4668-a2de-4a412c7bedf1 admin admin] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:30.039252 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c249330-a0e5-4668-a2de-4a412c7bedf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:30.039530 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c249330-a0e5-4668-a2de-4a412c7bedf1 admin admin] PATH_INFO: `/v3/users` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:30.040239 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c249330-a0e5-4668-a2de-4a412c7bedf1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:30.042242 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c249330-a0e5-4668-a2de-4a412c7bedf1 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:30.083280 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569c87bc-4256-40b2-84c5-8c18878f9bc1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9de0428abc8400aa338aba4f7c34c94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:30.083280 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569c87bc-4256-40b2-84c5-8c18878f9bc1 service cinder] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:30.083611 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569c87bc-4256-40b2-84c5-8c18878f9bc1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:30.084149 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569c87bc-4256-40b2-84c5-8c18878f9bc1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:30.086114 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-569c87bc-4256-40b2-84c5-8c18878f9bc1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:16:30.157742 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 810/1644] 10.222.0.23 () {60 vars in 1143 bytes} [Sat Jun 18 05:16:29 2022] POST /identity/v3/users => generated 308 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:16:30.161166 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c249330-a0e5-4668-a2de-4a412c7bedf1 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:30.183270 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569c87bc-4256-40b2-84c5-8c18878f9bc1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:30.183270 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569c87bc-4256-40b2-84c5-8c18878f9bc1 service cinder] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:30.200281 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c249330-a0e5-4668-a2de-4a412c7bedf1 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:30.207190 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 835/1645] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:16:29 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:16:30.322799 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53be9f2e-9ff3-47eb-b4f3-e90ffce97dbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:30.322799 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53be9f2e-9ff3-47eb-b4f3-e90ffce97dbd admin admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:30.322799 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53be9f2e-9ff3-47eb-b4f3-e90ffce97dbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:30.322799 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53be9f2e-9ff3-47eb-b4f3-e90ffce97dbd admin admin] PATH_INFO: `/v3/roles` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:30.323501 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53be9f2e-9ff3-47eb-b4f3-e90ffce97dbd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:30.323501 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53be9f2e-9ff3-47eb-b4f3-e90ffce97dbd admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:30.323811 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:16:30.323921 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 811/1646] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:16:30 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 18 05:16:30.324016 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569c87bc-4256-40b2-84c5-8c18878f9bc1 service cinder] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:30.324016 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569c87bc-4256-40b2-84c5-8c18878f9bc1 service cinder] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:30.324016 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5329e8fb-033b-4d6a-9445-6d8ef9f69911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:30.324016 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5329e8fb-033b-4d6a-9445-6d8ef9f69911 admin admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:30.324342 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5329e8fb-033b-4d6a-9445-6d8ef9f69911 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:30.324342 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5329e8fb-033b-4d6a-9445-6d8ef9f69911 admin admin] PATH_INFO: `/v3/projects/2ed2f69568d0430ab74917f326a3022f/users/12f9bc9d92d745858d0202adb53c4f16/roles/21c65f2c51bd4dd283d13507d2714467` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:30.381641 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5329e8fb-033b-4d6a-9445-6d8ef9f69911 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ed2f69568d0430ab74917f326a3022f, user_id=12f9bc9d92d745858d0202adb53c4f16, role_id=21c65f2c51bd4dd283d13507d2714467)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:30.410514 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5329e8fb-033b-4d6a-9445-6d8ef9f69911 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:30.518230 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 836/1647] 10.222.0.23 () {60 vars in 1482 bytes} [Sat Jun 18 05:16:30 2022] PUT /identity/v3/projects/2ed2f69568d0430ab74917f326a3022f/users/12f9bc9d92d745858d0202adb53c4f16/roles/21c65f2c51bd4dd283d13507d2714467 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:16:30.523284 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53be9f2e-9ff3-47eb-b4f3-e90ffce97dbd admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:30.545309 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53be9f2e-9ff3-47eb-b4f3-e90ffce97dbd admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:30.639811 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a03f13-9153-4ed0-9954-a8e3787ffb22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:30.640540 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a03f13-9153-4ed0-9954-a8e3787ffb22 admin admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:30.640831 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a03f13-9153-4ed0-9954-a8e3787ffb22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:30.641792 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a03f13-9153-4ed0-9954-a8e3787ffb22 admin admin] PATH_INFO: `/v3/roles` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:30.646301 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a03f13-9153-4ed0-9954-a8e3787ffb22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:30.646301 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a03f13-9153-4ed0-9954-a8e3787ffb22 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:30.659630 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 812/1648] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:16:30 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:16:30.666598 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5329e8fb-033b-4d6a-9445-6d8ef9f69911 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:30.717581 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5329e8fb-033b-4d6a-9445-6d8ef9f69911 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:30.748179 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b412c2d-7147-4a43-b42b-30cd80095315 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:30.749156 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b412c2d-7147-4a43-b42b-30cd80095315 admin admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:30.749475 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b412c2d-7147-4a43-b42b-30cd80095315 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:30.749760 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b412c2d-7147-4a43-b42b-30cd80095315 admin admin] PATH_INFO: `/v3/projects/2ed2f69568d0430ab74917f326a3022f/users/12f9bc9d92d745858d0202adb53c4f16/roles/21c65f2c51bd4dd283d13507d2714467` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:30.757621 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b412c2d-7147-4a43-b42b-30cd80095315 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ed2f69568d0430ab74917f326a3022f, user_id=12f9bc9d92d745858d0202adb53c4f16, role_id=21c65f2c51bd4dd283d13507d2714467)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:30.760328 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b412c2d-7147-4a43-b42b-30cd80095315 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:30.789866 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 837/1649] 10.222.0.23 () {60 vars in 1482 bytes} [Sat Jun 18 05:16:30 2022] PUT /identity/v3/projects/2ed2f69568d0430ab74917f326a3022f/users/12f9bc9d92d745858d0202adb53c4f16/roles/21c65f2c51bd4dd283d13507d2714467 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:16:31.166903 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a03f13-9153-4ed0-9954-a8e3787ffb22 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:31.171539 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-07a03f13-9153-4ed0-9954-a8e3787ffb22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:16:31.202182 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a03f13-9153-4ed0-9954-a8e3787ffb22 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:31.308461 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c030ccb7-d4e3-4849-b562-2866e321d309 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:31.309993 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c030ccb7-d4e3-4849-b562-2866e321d309 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:31.310435 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c030ccb7-d4e3-4849-b562-2866e321d309 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:31.310864 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c030ccb7-d4e3-4849-b562-2866e321d309 admin admin] PATH_INFO: `/v3/projects/c7a2526885554e8d8db2b050e8e47d53` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:31.325348 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c030ccb7-d4e3-4849-b562-2866e321d309 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c7a2526885554e8d8db2b050e8e47d53)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:31.330221 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c030ccb7-d4e3-4849-b562-2866e321d309 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:31.604698 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390b24e2-3399-41ab-85e9-7f936b9ac97f None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:31.604698 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390b24e2-3399-41ab-85e9-7f936b9ac97f None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:31.604698 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390b24e2-3399-41ab-85e9-7f936b9ac97f None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:31.604698 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 838/1650] 10.222.0.23 () {54 vars in 961 bytes} [Sat Jun 18 05:16:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 18 05:16:31.604698 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ded155-386f-4d1f-b7ef-76317f481308 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:31.604698 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ded155-386f-4d1f-b7ef-76317f481308 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:31.608473 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ded155-386f-4d1f-b7ef-76317f481308 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:31.612843 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:16:31.615510 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-36ded155-386f-4d1f-b7ef-76317f481308 None None] MFA Rules not processed for user `15e7f45ee0744c1d9602187eac8e8b80`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:16:31.615510 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-36ded155-386f-4d1f-b7ef-76317f481308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:16:31.615510 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 839/1651] 10.222.0.23 () {58 vars in 1058 bytes} [Sat Jun 18 05:16:31 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 18 05:16:31.615510 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb4ddd6-13ca-4bcb-b46b-d57076629e21 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:31.615510 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb4ddd6-13ca-4bcb-b46b-d57076629e21 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:31.615510 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb4ddd6-13ca-4bcb-b46b-d57076629e21 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:31.620088 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 840/1652] 10.222.0.23 () {54 vars in 912 bytes} [Sat Jun 18 05:16:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 18 05:16:31.620088 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faebf9c-d9a7-445d-95a9-74e7f4b8861d None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:31.620088 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faebf9c-d9a7-445d-95a9-74e7f4b8861d None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:31.620088 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faebf9c-d9a7-445d-95a9-74e7f4b8861d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:31.623147 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-c030ccb7-d4e3-4849-b562-2866e321d309 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c7a2526885554e8d8db2b050e8e47d53', 'request_id': 'req-c030ccb7-d4e3-4849-b562-2866e321d309'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:16:31.624698 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-8faebf9c-d9a7-445d-95a9-74e7f4b8861d None None] MFA Rules not processed for user `15e7f45ee0744c1d9602187eac8e8b80`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:16:31.625065 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-c030ccb7-d4e3-4849-b562-2866e321d309 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c7a2526885554e8d8db2b050e8e47d53', 'request_id': 'req-c030ccb7-d4e3-4849-b562-2866e321d309'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:16:31.625472 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-8faebf9c-d9a7-445d-95a9-74e7f4b8861d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:16:31.723537 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 841/1653] 10.222.0.23 () {58 vars in 1009 bytes} [Sat Jun 18 05:16:31 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 18 05:16:31.733241 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8faebf9c-d9a7-445d-95a9-74e7f4b8861d None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:31.757913 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-c030ccb7-d4e3-4849-b562-2866e321d309 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c7a2526885554e8d8db2b050e8e47d53', 'request_id': 'req-c030ccb7-d4e3-4849-b562-2866e321d309'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:16:31.774324 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-c030ccb7-d4e3-4849-b562-2866e321d309 admin admin] The token cache is being invalidate because project c7a2526885554e8d8db2b050e8e47d53 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80468) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 18 05:16:31.774633 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-c030ccb7-d4e3-4849-b562-2866e321d309 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:16:31.777521 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-c030ccb7-d4e3-4849-b562-2866e321d309 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:16:31.779999 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 813/1654] 10.222.0.23 () {58 vars in 1232 bytes} [Sat Jun 18 05:16:31 2022] DELETE /identity/v3/projects/c7a2526885554e8d8db2b050e8e47d53 => generated 0 bytes in 614 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:16:31.784269 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c030ccb7-d4e3-4849-b562-2866e321d309 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:31.790052 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-c030ccb7-d4e3-4849-b562-2866e321d309 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:16:31.827645 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8faebf9c-d9a7-445d-95a9-74e7f4b8861d None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:31.832168 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba24c8d0-c3af-4c49-b703-6f7100418de7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9de0428abc8400aa338aba4f7c34c94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:31.833007 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba24c8d0-c3af-4c49-b703-6f7100418de7 service cinder] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:31.833410 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba24c8d0-c3af-4c49-b703-6f7100418de7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:31.833826 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba24c8d0-c3af-4c49-b703-6f7100418de7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:31.837533 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-ba24c8d0-c3af-4c49-b703-6f7100418de7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:16:31.868012 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba24c8d0-c3af-4c49-b703-6f7100418de7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:31.869755 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba24c8d0-c3af-4c49-b703-6f7100418de7 service cinder] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:31.894011 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 842/1655] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:16:31 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 18 05:16:31.950807 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c030ccb7-d4e3-4849-b562-2866e321d309 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:31.958266 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89ca272-4296-4867-9ad6-80a20ff623ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:31.959479 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89ca272-4296-4867-9ad6-80a20ff623ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:31.961312 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89ca272-4296-4867-9ad6-80a20ff623ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:31.961879 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89ca272-4296-4867-9ad6-80a20ff623ba admin admin] PATH_INFO: `/v3/users/59c8f79e06ec4d32a0c7fe54308810f1` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:31.963432 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89ca272-4296-4867-9ad6-80a20ff623ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=59c8f79e06ec4d32a0c7fe54308810f1)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:31.965278 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: WARNING py.warnings [None req-d89ca272-4296-4867-9ad6-80a20ff623ba admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 18 05:16:31.965278 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: warnings.warn(msg) Jun 18 05:16:31.965278 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: Jun 18 05:16:31.967251 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89ca272-4296-4867-9ad6-80a20ff623ba admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:32.218413 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-d89ca272-4296-4867-9ad6-80a20ff623ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '59c8f79e06ec4d32a0c7fe54308810f1'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:16:32.219894 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-d89ca272-4296-4867-9ad6-80a20ff623ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '59c8f79e06ec4d32a0c7fe54308810f1'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:16:32.311469 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-d89ca272-4296-4867-9ad6-80a20ff623ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '59c8f79e06ec4d32a0c7fe54308810f1'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:16:32.367557 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-d89ca272-4296-4867-9ad6-80a20ff623ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '59c8f79e06ec4d32a0c7fe54308810f1'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:16:32.420965 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-d89ca272-4296-4867-9ad6-80a20ff623ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '59c8f79e06ec4d32a0c7fe54308810f1'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:16:32.448902 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 814/1656] 10.222.0.23 () {58 vars in 1223 bytes} [Sat Jun 18 05:16:31 2022] DELETE /identity/v3/users/59c8f79e06ec4d32a0c7fe54308810f1 => generated 0 bytes in 665 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:16:32.727845 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:16:33.257055 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:DELETE=2 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:16:33.723700 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:16:34.862262 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba24c8d0-c3af-4c49-b703-6f7100418de7 service cinder] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:34.881237 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba24c8d0-c3af-4c49-b703-6f7100418de7 service cinder] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:35.033562 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c71c90-58b8-47e3-a7fe-36035821fee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:35.034608 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c71c90-58b8-47e3-a7fe-36035821fee6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:35.035145 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c71c90-58b8-47e3-a7fe-36035821fee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:35.035490 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c71c90-58b8-47e3-a7fe-36035821fee6 admin admin] PATH_INFO: `/v3/projects/0611875730fb403c95901c0c9de52aa5` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:35.037926 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c71c90-58b8-47e3-a7fe-36035821fee6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0611875730fb403c95901c0c9de52aa5)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:35.041606 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: WARNING py.warnings [None req-79c71c90-58b8-47e3-a7fe-36035821fee6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 18 05:16:35.041606 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: warnings.warn(msg) Jun 18 05:16:35.041606 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: Jun 18 05:16:35.043125 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c71c90-58b8-47e3-a7fe-36035821fee6 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:35.367719 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-79c71c90-58b8-47e3-a7fe-36035821fee6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0611875730fb403c95901c0c9de52aa5', 'request_id': 'req-79c71c90-58b8-47e3-a7fe-36035821fee6'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:16:35.370673 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c88ac3d-b406-439b-8965-4ef845079f40 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:35.371562 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c88ac3d-b406-439b-8965-4ef845079f40 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:35.372157 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c88ac3d-b406-439b-8965-4ef845079f40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:35.388136 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-79c71c90-58b8-47e3-a7fe-36035821fee6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0611875730fb403c95901c0c9de52aa5', 'request_id': 'req-79c71c90-58b8-47e3-a7fe-36035821fee6'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:16:35.456560 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-79c71c90-58b8-47e3-a7fe-36035821fee6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0611875730fb403c95901c0c9de52aa5', 'request_id': 'req-79c71c90-58b8-47e3-a7fe-36035821fee6'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:16:35.468863 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-4c88ac3d-b406-439b-8965-4ef845079f40 None None] MFA Rules not processed for user `12f9bc9d92d745858d0202adb53c4f16`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:16:35.471841 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-4c88ac3d-b406-439b-8965-4ef845079f40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:16:35.546320 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-79c71c90-58b8-47e3-a7fe-36035821fee6 admin admin] The token cache is being invalidate because project 0611875730fb403c95901c0c9de52aa5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80469) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 18 05:16:35.553277 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-79c71c90-58b8-47e3-a7fe-36035821fee6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:16:35.553277 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-79c71c90-58b8-47e3-a7fe-36035821fee6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:16:35.560543 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 843/1657] 10.222.0.23 () {58 vars in 1232 bytes} [Sat Jun 18 05:16:34 2022] DELETE /identity/v3/projects/0611875730fb403c95901c0c9de52aa5 => generated 0 bytes in 694 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:16:35.562111 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e649f2-8078-49e6-ae57-9cfbfc3e15fa None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:35.562410 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e649f2-8078-49e6-ae57-9cfbfc3e15fa None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:35.562703 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e649f2-8078-49e6-ae57-9cfbfc3e15fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:35.597069 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-31e649f2-8078-49e6-ae57-9cfbfc3e15fa None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:16:35.600745 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:16:35.602204 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-31e649f2-8078-49e6-ae57-9cfbfc3e15fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:16:35.708781 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 815/1658] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:16:35 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 335 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:16:35.751629 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 844/1659] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:16:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:16:35.760544 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfcbc99-daaa-45e7-83ef-d6bf85f807fb None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:35.760544 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfcbc99-daaa-45e7-83ef-d6bf85f807fb None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:35.760845 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfcbc99-daaa-45e7-83ef-d6bf85f807fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:35.796184 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-1cfcbc99-daaa-45e7-83ef-d6bf85f807fb None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:16:35.798163 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-1cfcbc99-daaa-45e7-83ef-d6bf85f807fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:16:35.828641 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 816/1660] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:16:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:16:35.831433 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e649f2-8078-49e6-ae57-9cfbfc3e15fa None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:35.837989 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:16:35.849508 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e649f2-8078-49e6-ae57-9cfbfc3e15fa None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:35.857443 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ef55c4-bdd9-455d-8403-7a33b43f59cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:35.858191 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ef55c4-bdd9-455d-8403-7a33b43f59cb admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:35.858623 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ef55c4-bdd9-455d-8403-7a33b43f59cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:35.858808 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ef55c4-bdd9-455d-8403-7a33b43f59cb admin admin] PATH_INFO: `/v3/domains` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:35.859553 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ef55c4-bdd9-455d-8403-7a33b43f59cb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 05:16:35.859952 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ef55c4-bdd9-455d-8403-7a33b43f59cb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:35.861510 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ef55c4-bdd9-455d-8403-7a33b43f59cb admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:35.921211 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfcbc99-daaa-45e7-83ef-d6bf85f807fb None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:35.928070 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 845/1661] 10.222.0.23 () {58 vars in 1152 bytes} [Sat Jun 18 05:16:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 05:16:35.934907 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfcbc99-daaa-45e7-83ef-d6bf85f807fb None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:35.951856 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8525189e-6d26-41e4-ae3f-7ebe77008cc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:35.953544 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8525189e-6d26-41e4-ae3f-7ebe77008cc1 admin admin] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:35.953959 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8525189e-6d26-41e4-ae3f-7ebe77008cc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:35.954353 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8525189e-6d26-41e4-ae3f-7ebe77008cc1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:35.955693 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8525189e-6d26-41e4-ae3f-7ebe77008cc1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:35.958368 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8525189e-6d26-41e4-ae3f-7ebe77008cc1 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:36.079453 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 817/1662] 10.222.0.23 () {60 vars in 1152 bytes} [Sat Jun 18 05:16:35 2022] POST /identity/v3/projects => generated 387 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 18 05:16:36.083372 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ef55c4-bdd9-455d-8403-7a33b43f59cb admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:36.105450 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ef55c4-bdd9-455d-8403-7a33b43f59cb admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:36.289337 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d49b1c4-a65a-4a67-87d2-727391dbf13d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:36.290322 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d49b1c4-a65a-4a67-87d2-727391dbf13d admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:36.290762 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d49b1c4-a65a-4a67-87d2-727391dbf13d admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:36.291193 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d49b1c4-a65a-4a67-87d2-727391dbf13d admin admin] PATH_INFO: `/v3/users` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:36.294280 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d49b1c4-a65a-4a67-87d2-727391dbf13d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:36.296836 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d49b1c4-a65a-4a67-87d2-727391dbf13d admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:36.407488 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 846/1663] 10.222.0.23 () {60 vars in 1143 bytes} [Sat Jun 18 05:16:36 2022] POST /identity/v3/users => generated 294 bytes in 324 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:16:36.411304 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8525189e-6d26-41e4-ae3f-7ebe77008cc1 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:36.448235 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8525189e-6d26-41e4-ae3f-7ebe77008cc1 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:36.463321 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b08492-5bea-449f-a139-a19de9f0808e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:36.464180 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b08492-5bea-449f-a139-a19de9f0808e admin admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:36.465967 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b08492-5bea-449f-a139-a19de9f0808e admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:36.466392 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b08492-5bea-449f-a139-a19de9f0808e admin admin] PATH_INFO: `/v3/roles` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:36.471044 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b08492-5bea-449f-a139-a19de9f0808e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:36.476025 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b08492-5bea-449f-a139-a19de9f0808e admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:36.494391 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 818/1664] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:16:36 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:16:36.501794 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d49b1c4-a65a-4a67-87d2-727391dbf13d admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:36.529297 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d49b1c4-a65a-4a67-87d2-727391dbf13d admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:36.540013 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0b2709-0b0e-405a-be6f-21e2dc16ca69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:36.543683 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0b2709-0b0e-405a-be6f-21e2dc16ca69 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:36.544211 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0b2709-0b0e-405a-be6f-21e2dc16ca69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:36.545777 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0b2709-0b0e-405a-be6f-21e2dc16ca69 admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:36.546752 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0b2709-0b0e-405a-be6f-21e2dc16ca69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:36.556387 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0b2709-0b0e-405a-be6f-21e2dc16ca69 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:36.586960 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 847/1665] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:16:36 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:16:36.591354 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b08492-5bea-449f-a139-a19de9f0808e admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:36.603519 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b08492-5bea-449f-a139-a19de9f0808e admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:36.610177 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f117d03-cca2-49f0-83da-1fbf94d89ca8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:36.611226 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f117d03-cca2-49f0-83da-1fbf94d89ca8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:36.612398 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f117d03-cca2-49f0-83da-1fbf94d89ca8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:36.613661 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f117d03-cca2-49f0-83da-1fbf94d89ca8 admin admin] PATH_INFO: `/v3/projects/1ba30ec379cf4dc3bd1789f42bf7295a/users/061360a551f34be58214907299ae9a26/roles/cbf5c4c26898413d93e73b46b41046a3` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:36.665273 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f117d03-cca2-49f0-83da-1fbf94d89ca8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ba30ec379cf4dc3bd1789f42bf7295a, user_id=061360a551f34be58214907299ae9a26, role_id=cbf5c4c26898413d93e73b46b41046a3)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:36.668909 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f117d03-cca2-49f0-83da-1fbf94d89ca8 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:36.753645 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 819/1666] 10.222.0.23 () {60 vars in 1482 bytes} [Sat Jun 18 05:16:36 2022] PUT /identity/v3/projects/1ba30ec379cf4dc3bd1789f42bf7295a/users/061360a551f34be58214907299ae9a26/roles/cbf5c4c26898413d93e73b46b41046a3 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:16:36.765180 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9054afc-ab94-4511-8721-9ba69b621598 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:36.765623 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9054afc-ab94-4511-8721-9ba69b621598 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:36.766119 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9054afc-ab94-4511-8721-9ba69b621598 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:36.858254 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-b9054afc-ab94-4511-8721-9ba69b621598 None None] MFA Rules not processed for user `061360a551f34be58214907299ae9a26`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:16:36.861853 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-b9054afc-ab94-4511-8721-9ba69b621598 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:16:36.994891 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 848/1667] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:16:36 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:16:36.999368 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f117d03-cca2-49f0-83da-1fbf94d89ca8 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:37.189436 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f117d03-cca2-49f0-83da-1fbf94d89ca8 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:37.204851 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af038e05-bfa5-4e79-9827-53c76c1d36a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:37.205649 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af038e05-bfa5-4e79-9827-53c76c1d36a7 admin admin] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:37.205941 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af038e05-bfa5-4e79-9827-53c76c1d36a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:37.206243 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af038e05-bfa5-4e79-9827-53c76c1d36a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:37.206966 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af038e05-bfa5-4e79-9827-53c76c1d36a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:37.208893 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af038e05-bfa5-4e79-9827-53c76c1d36a7 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:37.323930 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 820/1668] 10.222.0.23 () {60 vars in 1152 bytes} [Sat Jun 18 05:16:36 2022] POST /identity/v3/projects => generated 387 bytes in 325 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:16:37.327879 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9054afc-ab94-4511-8721-9ba69b621598 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:37.346527 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9054afc-ab94-4511-8721-9ba69b621598 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:37.463736 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:16:37.477060 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8fc59d-710b-4c0a-a86d-3f7ebb316b47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:37.478446 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8fc59d-710b-4c0a-a86d-3f7ebb316b47 admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:37.478446 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8fc59d-710b-4c0a-a86d-3f7ebb316b47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:37.478446 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8fc59d-710b-4c0a-a86d-3f7ebb316b47 admin admin] PATH_INFO: `/v3/users` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:37.479077 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8fc59d-710b-4c0a-a86d-3f7ebb316b47 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:37.481115 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8fc59d-710b-4c0a-a86d-3f7ebb316b47 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:37.661885 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 849/1669] 10.222.0.23 () {60 vars in 1143 bytes} [Sat Jun 18 05:16:37 2022] POST /identity/v3/users => generated 300 bytes in 334 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:16:37.667553 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af038e05-bfa5-4e79-9827-53c76c1d36a7 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:37.682517 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af038e05-bfa5-4e79-9827-53c76c1d36a7 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:37.696401 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f453a90c-857e-43ea-b0d2-eb89b39c682b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:37.697097 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f453a90c-857e-43ea-b0d2-eb89b39c682b admin admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:37.697402 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f453a90c-857e-43ea-b0d2-eb89b39c682b admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:37.697696 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f453a90c-857e-43ea-b0d2-eb89b39c682b admin admin] PATH_INFO: `/v3/roles` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:37.698487 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f453a90c-857e-43ea-b0d2-eb89b39c682b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:37.700136 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f453a90c-857e-43ea-b0d2-eb89b39c682b admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:37.743304 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 821/1670] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:16:37 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:16:37.746422 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8fc59d-710b-4c0a-a86d-3f7ebb316b47 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:37.759092 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8fc59d-710b-4c0a-a86d-3f7ebb316b47 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:37.782281 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cefaf37-6574-4fba-9175-d20ac4ff4fb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:37.783183 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cefaf37-6574-4fba-9175-d20ac4ff4fb3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:37.783499 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cefaf37-6574-4fba-9175-d20ac4ff4fb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:37.783817 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cefaf37-6574-4fba-9175-d20ac4ff4fb3 admin admin] PATH_INFO: `/v3/projects/a684f86fb5554337a8ba861ffb29239b/users/f31b4faefebd4d0baeb6840e3166513f/roles/21c65f2c51bd4dd283d13507d2714467` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:37.825861 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cefaf37-6574-4fba-9175-d20ac4ff4fb3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a684f86fb5554337a8ba861ffb29239b, user_id=f31b4faefebd4d0baeb6840e3166513f, role_id=21c65f2c51bd4dd283d13507d2714467)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:37.835146 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cefaf37-6574-4fba-9175-d20ac4ff4fb3 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:37.955300 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 850/1671] 10.222.0.23 () {60 vars in 1482 bytes} [Sat Jun 18 05:16:37 2022] PUT /identity/v3/projects/a684f86fb5554337a8ba861ffb29239b/users/f31b4faefebd4d0baeb6840e3166513f/roles/21c65f2c51bd4dd283d13507d2714467 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:16:37.956288 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f453a90c-857e-43ea-b0d2-eb89b39c682b admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:37.973838 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f453a90c-857e-43ea-b0d2-eb89b39c682b admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:37.984508 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125772ac-b829-489f-945e-20716732a23d None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:37.984872 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125772ac-b829-489f-945e-20716732a23d None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:37.985319 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125772ac-b829-489f-945e-20716732a23d None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:37.987522 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 851/1672] 10.222.0.23 () {54 vars in 921 bytes} [Sat Jun 18 05:16:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 05:16:38.016105 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b55a46-68c8-448c-9252-ab3eebfc5015 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:38.016105 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b55a46-68c8-448c-9252-ab3eebfc5015 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:38.016105 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b55a46-68c8-448c-9252-ab3eebfc5015 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:38.036132 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:16:38.057676 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-04b55a46-68c8-448c-9252-ab3eebfc5015 None None] MFA Rules not processed for user `5323ffd91f864680b0c0084eb3f44d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:16:38.059895 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-04b55a46-68c8-448c-9252-ab3eebfc5015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:16:38.105587 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a8208d-02ac-4fca-8161-e41824a1cd14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:38.107472 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a8208d-02ac-4fca-8161-e41824a1cd14 admin admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:38.110586 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a8208d-02ac-4fca-8161-e41824a1cd14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:38.111139 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a8208d-02ac-4fca-8161-e41824a1cd14 admin admin] PATH_INFO: `/v3/roles` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:38.113052 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a8208d-02ac-4fca-8161-e41824a1cd14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:38.118144 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a8208d-02ac-4fca-8161-e41824a1cd14 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:38.182544 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 822/1673] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:16:37 2022] GET /identity/v3/roles => generated 1420 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:16:38.184984 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a8208d-02ac-4fca-8161-e41824a1cd14 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:38.208480 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a8208d-02ac-4fca-8161-e41824a1cd14 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:38.221126 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83a5657-ad91-4f96-8300-c2ced8ef0227 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:38.224728 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83a5657-ad91-4f96-8300-c2ced8ef0227 admin admin] REQUEST_METHOD: `PUT` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:38.225820 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83a5657-ad91-4f96-8300-c2ced8ef0227 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:38.226526 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83a5657-ad91-4f96-8300-c2ced8ef0227 admin admin] PATH_INFO: `/v3/projects/a684f86fb5554337a8ba861ffb29239b/users/f31b4faefebd4d0baeb6840e3166513f/roles/21c65f2c51bd4dd283d13507d2714467` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:38.242850 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83a5657-ad91-4f96-8300-c2ced8ef0227 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a684f86fb5554337a8ba861ffb29239b, user_id=f31b4faefebd4d0baeb6840e3166513f, role_id=21c65f2c51bd4dd283d13507d2714467)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:38.243598 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 852/1674] 10.222.0.23 () {58 vars in 1018 bytes} [Sat Jun 18 05:16:38 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:16:38.250507 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83a5657-ad91-4f96-8300-c2ced8ef0227 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:38.282231 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 823/1675] 10.222.0.23 () {60 vars in 1482 bytes} [Sat Jun 18 05:16:38 2022] PUT /identity/v3/projects/a684f86fb5554337a8ba861ffb29239b/users/f31b4faefebd4d0baeb6840e3166513f/roles/21c65f2c51bd4dd283d13507d2714467 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 18 05:16:38.294285 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c897533-2734-460b-8445-08ee143b0fb1 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:38.294575 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c897533-2734-460b-8445-08ee143b0fb1 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:38.298255 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c897533-2734-460b-8445-08ee143b0fb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:38.333247 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-9c897533-2734-460b-8445-08ee143b0fb1 None None] MFA Rules not processed for user `f31b4faefebd4d0baeb6840e3166513f`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:16:38.334698 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-9c897533-2734-460b-8445-08ee143b0fb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:16:38.351098 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e1fd02-c59a-40a1-b4c8-1a9750b56307 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:38.351390 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e1fd02-c59a-40a1-b4c8-1a9750b56307 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:38.351983 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e1fd02-c59a-40a1-b4c8-1a9750b56307 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:38.436250 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-f5e1fd02-c59a-40a1-b4c8-1a9750b56307 None None] MFA Rules not processed for user `79f8817a70b844d7ac49543d74d0e38b`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:16:38.437965 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-f5e1fd02-c59a-40a1-b4c8-1a9750b56307 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:16:38.539741 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 853/1676] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:16:38 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:16:38.541511 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1f328c-5752-4fc6-9bfa-a3d40581a0ff None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:38.542916 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1f328c-5752-4fc6-9bfa-a3d40581a0ff None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:38.543550 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1f328c-5752-4fc6-9bfa-a3d40581a0ff None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:38.565630 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 854/1677] 10.222.0.23 () {54 vars in 960 bytes} [Sat Jun 18 05:16:38 2022] GET /identity => generated 270 bytes in 24 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 05:16:38.567367 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1f328c-5752-4fc6-9bfa-a3d40581a0ff None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:38.572892 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-2a1f328c-5752-4fc6-9bfa-a3d40581a0ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:16:38.604405 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 824/1678] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:16:38 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 18 05:16:38.619532 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7e7f9c-bd3b-4787-8364-fbd4ad1d8561 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:38.619942 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7e7f9c-bd3b-4787-8364-fbd4ad1d8561 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:38.620236 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7e7f9c-bd3b-4787-8364-fbd4ad1d8561 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:38.685945 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-ae7e7f9c-bd3b-4787-8364-fbd4ad1d8561 None None] MFA Rules not processed for user `5323ffd91f864680b0c0084eb3f44d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:16:38.693251 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-ae7e7f9c-bd3b-4787-8364-fbd4ad1d8561 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:16:38.707168 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:16:38.748231 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1f328c-5752-4fc6-9bfa-a3d40581a0ff None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:38.759961 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af0c1da-7dcc-4d52-85af-516886638083 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9de0428abc8400aa338aba4f7c34c94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:38.760769 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af0c1da-7dcc-4d52-85af-516886638083 service cinder] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:38.761109 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af0c1da-7dcc-4d52-85af-516886638083 service cinder] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:38.761476 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af0c1da-7dcc-4d52-85af-516886638083 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:38.787246 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af0c1da-7dcc-4d52-85af-516886638083 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:38.788902 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af0c1da-7dcc-4d52-85af-516886638083 service cinder] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:38.833270 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 855/1679] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:16:38 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:16:38.838182 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411b5199-a59c-48df-b96f-19c66ea05448 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:38.838908 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411b5199-a59c-48df-b96f-19c66ea05448 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:38.839226 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411b5199-a59c-48df-b96f-19c66ea05448 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:38.861463 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-411b5199-a59c-48df-b96f-19c66ea05448 None None] MFA Rules not processed for user `79f8817a70b844d7ac49543d74d0e38b`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:16:38.862801 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-411b5199-a59c-48df-b96f-19c66ea05448 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:16:38.877941 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 856/1680] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:16:38 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:16:38.888174 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 825/1681] 10.222.0.23 () {58 vars in 1057 bytes} [Sat Jun 18 05:16:38 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:16:38.899690 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411b5199-a59c-48df-b96f-19c66ea05448 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:38.915880 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411b5199-a59c-48df-b96f-19c66ea05448 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:38.922423 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680d6f2e-66ad-493a-a4f8-504f653bf61f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5323ffd91f864680b0c0084eb3f44d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:38.923227 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680d6f2e-66ad-493a-a4f8-504f653bf61f service placement] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:38.923519 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680d6f2e-66ad-493a-a4f8-504f653bf61f service placement] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:38.923798 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680d6f2e-66ad-493a-a4f8-504f653bf61f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:38.936825 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680d6f2e-66ad-493a-a4f8-504f653bf61f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:38.938959 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680d6f2e-66ad-493a-a4f8-504f653bf61f service placement] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:38.953209 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 857/1682] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:16:38 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:16:39.088658 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7e7f9c-bd3b-4787-8364-fbd4ad1d8561 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:39.090456 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-ae7e7f9c-bd3b-4787-8364-fbd4ad1d8561 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:16:39.176313 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:16:39.221943 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7e7f9c-bd3b-4787-8364-fbd4ad1d8561 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:39.228990 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279b3ffb-e4a7-4ec1-9119-1cf4c079c5a9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15e7f45ee0744c1d9602187eac8e8b80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:39.230772 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279b3ffb-e4a7-4ec1-9119-1cf4c079c5a9 service nova] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:39.231692 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279b3ffb-e4a7-4ec1-9119-1cf4c079c5a9 service nova] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:39.232092 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279b3ffb-e4a7-4ec1-9119-1cf4c079c5a9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:39.243146 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279b3ffb-e4a7-4ec1-9119-1cf4c079c5a9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:39.245177 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279b3ffb-e4a7-4ec1-9119-1cf4c079c5a9 service nova] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:39.272325 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 826/1683] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:16:39 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:16:39.588243 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680d6f2e-66ad-493a-a4f8-504f653bf61f service placement] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:39.742822 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680d6f2e-66ad-493a-a4f8-504f653bf61f service placement] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:39.755106 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1da0f44-5a8b-417f-ab0b-c6f516ae78d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:39.756063 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1da0f44-5a8b-417f-ab0b-c6f516ae78d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:39.756542 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1da0f44-5a8b-417f-ab0b-c6f516ae78d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:39.756829 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1da0f44-5a8b-417f-ab0b-c6f516ae78d9 admin admin] PATH_INFO: `/v3/users/061360a551f34be58214907299ae9a26` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:39.762140 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1da0f44-5a8b-417f-ab0b-c6f516ae78d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=061360a551f34be58214907299ae9a26)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:39.764201 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1da0f44-5a8b-417f-ab0b-c6f516ae78d9 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:40.025417 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-e1da0f44-5a8b-417f-ab0b-c6f516ae78d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '061360a551f34be58214907299ae9a26'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:16:40.057110 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-e1da0f44-5a8b-417f-ab0b-c6f516ae78d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '061360a551f34be58214907299ae9a26'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:16:40.058462 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-e1da0f44-5a8b-417f-ab0b-c6f516ae78d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '061360a551f34be58214907299ae9a26'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:16:40.135046 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-e1da0f44-5a8b-417f-ab0b-c6f516ae78d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '061360a551f34be58214907299ae9a26'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:16:40.148971 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-e1da0f44-5a8b-417f-ab0b-c6f516ae78d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '061360a551f34be58214907299ae9a26'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:16:40.174707 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 858/1684] 10.222.0.23 () {58 vars in 1223 bytes} [Sat Jun 18 05:16:39 2022] DELETE /identity/v3/users/061360a551f34be58214907299ae9a26 => generated 0 bytes in 591 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:16:40.176023 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0768088-42d2-4d95-a1e8-496d1049d296 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:40.176497 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0768088-42d2-4d95-a1e8-496d1049d296 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:40.176976 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0768088-42d2-4d95-a1e8-496d1049d296 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:40.230529 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-d0768088-42d2-4d95-a1e8-496d1049d296 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:16:40.235629 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-d0768088-42d2-4d95-a1e8-496d1049d296 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:16:40.447833 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 827/1685] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:16:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:16:40.466953 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1da0f44-5a8b-417f-ab0b-c6f516ae78d9 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:40.472820 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-e1da0f44-5a8b-417f-ab0b-c6f516ae78d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:16:40.504259 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1da0f44-5a8b-417f-ab0b-c6f516ae78d9 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:40.534171 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dcec20-6943-4ece-9d87-76e8989ad169 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:40.535619 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dcec20-6943-4ece-9d87-76e8989ad169 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:40.536107 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dcec20-6943-4ece-9d87-76e8989ad169 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:40.538826 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dcec20-6943-4ece-9d87-76e8989ad169 admin admin] PATH_INFO: `/v3/users/7cad75680cbb451fad629ee0e0e3d536` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:40.546021 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41dcec20-6943-4ece-9d87-76e8989ad169 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7cad75680cbb451fad629ee0e0e3d536)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:40.548406 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41dcec20-6943-4ece-9d87-76e8989ad169 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:40.554259 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0768088-42d2-4d95-a1e8-496d1049d296 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:40.555903 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-d0768088-42d2-4d95-a1e8-496d1049d296 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:16:40.748580 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0768088-42d2-4d95-a1e8-496d1049d296 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:40.768812 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f903f3-6f9c-47e6-bbb6-513c7b17ca17 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c4d7cde8f4ad9aa479bb09e325db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:40.769536 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f903f3-6f9c-47e6-bbb6-513c7b17ca17 service neutron] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:40.769829 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f903f3-6f9c-47e6-bbb6-513c7b17ca17 service neutron] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:40.770109 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f903f3-6f9c-47e6-bbb6-513c7b17ca17 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:40.806287 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f903f3-6f9c-47e6-bbb6-513c7b17ca17 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:40.809299 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f903f3-6f9c-47e6-bbb6-513c7b17ca17 service neutron] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:40.835320 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 828/1686] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:16:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 18 05:16:40.891786 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:16:40.897840 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-41dcec20-6943-4ece-9d87-76e8989ad169 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7cad75680cbb451fad629ee0e0e3d536'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:16:40.911801 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-41dcec20-6943-4ece-9d87-76e8989ad169 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7cad75680cbb451fad629ee0e0e3d536'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:16:40.912574 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-41dcec20-6943-4ece-9d87-76e8989ad169 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7cad75680cbb451fad629ee0e0e3d536'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:16:40.937037 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39442ac1-b899-45fe-b666-abf58a4ec2a1 None None] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:40.937744 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39442ac1-b899-45fe-b666-abf58a4ec2a1 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:40.939089 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39442ac1-b899-45fe-b666-abf58a4ec2a1 None None] PATH_INFO: `/` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:40.941667 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 829/1687] 10.222.0.23 () {54 vars in 912 bytes} [Sat Jun 18 05:16:40 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 18 05:16:40.953712 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c4362f-eb95-4398-8f62-88825f61bb37 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:40.953712 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c4362f-eb95-4398-8f62-88825f61bb37 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:40.953712 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c4362f-eb95-4398-8f62-88825f61bb37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:41.007418 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-41dcec20-6943-4ece-9d87-76e8989ad169 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7cad75680cbb451fad629ee0e0e3d536'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:16:41.016596 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-41dcec20-6943-4ece-9d87-76e8989ad169 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7cad75680cbb451fad629ee0e0e3d536'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:16:41.037191 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 859/1688] 10.222.0.23 () {58 vars in 1223 bytes} [Sat Jun 18 05:16:40 2022] DELETE /identity/v3/users/7cad75680cbb451fad629ee0e0e3d536 => generated 0 bytes in 568 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:16:41.080969 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-27c4362f-eb95-4398-8f62-88825f61bb37 None None] MFA Rules not processed for user `e78c4d7cde8f4ad9aa479bb09e325db3`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:16:41.089795 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-27c4362f-eb95-4398-8f62-88825f61bb37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:16:41.211514 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 830/1689] 10.222.0.23 () {58 vars in 1009 bytes} [Sat Jun 18 05:16:40 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 18 05:16:41.236181 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dcec20-6943-4ece-9d87-76e8989ad169 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:41.238279 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-41dcec20-6943-4ece-9d87-76e8989ad169 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:16:41.270296 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dcec20-6943-4ece-9d87-76e8989ad169 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:41.282224 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb4ee05-b53f-46a2-b056-d9c1184664b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c4d7cde8f4ad9aa479bb09e325db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:41.283070 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb4ee05-b53f-46a2-b056-d9c1184664b2 service neutron] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:41.283543 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb4ee05-b53f-46a2-b056-d9c1184664b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:41.283831 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb4ee05-b53f-46a2-b056-d9c1184664b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:41.296570 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c98363-4cba-4605-8906-db6e9a423a76 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:41.297204 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c98363-4cba-4605-8906-db6e9a423a76 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:41.297693 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c98363-4cba-4605-8906-db6e9a423a76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:41.311924 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb4ee05-b53f-46a2-b056-d9c1184664b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:41.313680 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb4ee05-b53f-46a2-b056-d9c1184664b2 service neutron] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:41.345019 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 860/1690] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:16:41 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:16:41.346005 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:16:41.373226 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-17c98363-4cba-4605-8906-db6e9a423a76 None None] MFA Rules not processed for user `d2c8f964ee6a49cda501ed28a9432c8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:16:41.374589 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-17c98363-4cba-4605-8906-db6e9a423a76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:16:41.585119 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 831/1691] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:16:41 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:16:41.608627 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb4ee05-b53f-46a2-b056-d9c1184664b2 service neutron] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:41.610641 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-5bb4ee05-b53f-46a2-b056-d9c1184664b2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:16:41.811861 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb4ee05-b53f-46a2-b056-d9c1184664b2 service neutron] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:41.835400 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b431e17-c4a3-4481-a232-a08b1f9c7fc7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15e7f45ee0744c1d9602187eac8e8b80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:41.835400 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b431e17-c4a3-4481-a232-a08b1f9c7fc7 service nova] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:41.835400 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b431e17-c4a3-4481-a232-a08b1f9c7fc7 service nova] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:41.835400 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b431e17-c4a3-4481-a232-a08b1f9c7fc7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:41.865527 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b431e17-c4a3-4481-a232-a08b1f9c7fc7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:41.872622 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b431e17-c4a3-4481-a232-a08b1f9c7fc7 service nova] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:41.899830 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 861/1692] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:16:41 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:16:41.931996 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c98363-4cba-4605-8906-db6e9a423a76 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:41.933725 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-17c98363-4cba-4605-8906-db6e9a423a76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:16:42.076167 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c98363-4cba-4605-8906-db6e9a423a76 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:42.091331 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc4278e-b27c-440d-aec0-cfee26319e7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:42.096280 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b431e17-c4a3-4481-a232-a08b1f9c7fc7 service nova] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:42.098878 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-9b431e17-c4a3-4481-a232-a08b1f9c7fc7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:16:42.108695 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc4278e-b27c-440d-aec0-cfee26319e7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:42.112780 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc4278e-b27c-440d-aec0-cfee26319e7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:42.114335 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc4278e-b27c-440d-aec0-cfee26319e7c admin admin] PATH_INFO: `/v3/projects/780f29e257644af280e61247d02a3c1e` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:42.123706 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc4278e-b27c-440d-aec0-cfee26319e7c admin admin] RBAC: Authorizing `identity:delete_project(project_id=780f29e257644af280e61247d02a3c1e)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:42.151635 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc4278e-b27c-440d-aec0-cfee26319e7c admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:42.156637 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b431e17-c4a3-4481-a232-a08b1f9c7fc7 service nova] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:42.173854 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88884e3-bb1f-48b3-a312-8effce7270e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:42.187526 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88884e3-bb1f-48b3-a312-8effce7270e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:42.187526 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88884e3-bb1f-48b3-a312-8effce7270e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:42.187526 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88884e3-bb1f-48b3-a312-8effce7270e9 admin admin] PATH_INFO: `/v3/projects/1ba30ec379cf4dc3bd1789f42bf7295a` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:42.190358 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88884e3-bb1f-48b3-a312-8effce7270e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ba30ec379cf4dc3bd1789f42bf7295a)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:42.200410 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88884e3-bb1f-48b3-a312-8effce7270e9 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:42.424140 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-9cc4278e-b27c-440d-aec0-cfee26319e7c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '780f29e257644af280e61247d02a3c1e', 'request_id': 'req-9cc4278e-b27c-440d-aec0-cfee26319e7c'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:16:42.435585 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-e88884e3-bb1f-48b3-a312-8effce7270e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ba30ec379cf4dc3bd1789f42bf7295a', 'request_id': 'req-e88884e3-bb1f-48b3-a312-8effce7270e9'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:16:42.450140 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-9cc4278e-b27c-440d-aec0-cfee26319e7c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '780f29e257644af280e61247d02a3c1e', 'request_id': 'req-9cc4278e-b27c-440d-aec0-cfee26319e7c'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:16:42.463015 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-e88884e3-bb1f-48b3-a312-8effce7270e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ba30ec379cf4dc3bd1789f42bf7295a', 'request_id': 'req-e88884e3-bb1f-48b3-a312-8effce7270e9'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:16:42.493708 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-e88884e3-bb1f-48b3-a312-8effce7270e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ba30ec379cf4dc3bd1789f42bf7295a', 'request_id': 'req-e88884e3-bb1f-48b3-a312-8effce7270e9'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:16:42.568453 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-9cc4278e-b27c-440d-aec0-cfee26319e7c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '780f29e257644af280e61247d02a3c1e', 'request_id': 'req-9cc4278e-b27c-440d-aec0-cfee26319e7c'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:16:42.597025 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-9cc4278e-b27c-440d-aec0-cfee26319e7c admin admin] The token cache is being invalidate because project 780f29e257644af280e61247d02a3c1e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80468) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 18 05:16:42.604480 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-9cc4278e-b27c-440d-aec0-cfee26319e7c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:16:42.604480 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-9cc4278e-b27c-440d-aec0-cfee26319e7c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:16:42.604480 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 832/1693] 10.222.0.23 () {58 vars in 1232 bytes} [Sat Jun 18 05:16:41 2022] DELETE /identity/v3/projects/780f29e257644af280e61247d02a3c1e => generated 0 bytes in 672 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:16:42.610349 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d2bb5d-de8a-44b7-8e3a-4db312f4aaa5 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:42.610349 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d2bb5d-de8a-44b7-8e3a-4db312f4aaa5 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:42.612369 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d2bb5d-de8a-44b7-8e3a-4db312f4aaa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:42.625057 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-e88884e3-bb1f-48b3-a312-8effce7270e9 admin admin] The token cache is being invalidate because project 1ba30ec379cf4dc3bd1789f42bf7295a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80469) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 18 05:16:42.625353 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-e88884e3-bb1f-48b3-a312-8effce7270e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:16:42.626898 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-e88884e3-bb1f-48b3-a312-8effce7270e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:16:42.632024 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 862/1694] 10.222.0.23 () {58 vars in 1232 bytes} [Sat Jun 18 05:16:42 2022] DELETE /identity/v3/projects/1ba30ec379cf4dc3bd1789f42bf7295a => generated 0 bytes in 534 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:16:42.632024 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88884e3-bb1f-48b3-a312-8effce7270e9 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:42.656414 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-e88884e3-bb1f-48b3-a312-8effce7270e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:16:42.746234 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:16:42.761438 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-f2d2bb5d-de8a-44b7-8e3a-4db312f4aaa5 None None] MFA Rules not processed for user `e0f3999a062e45748878a3204e3ea456`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:16:42.765230 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-f2d2bb5d-de8a-44b7-8e3a-4db312f4aaa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:16:42.829111 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:16:42.890135 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88884e3-bb1f-48b3-a312-8effce7270e9 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:42.899399 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f1f3cc-aa08-4eca-8d39-3e18675536ad service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5323ffd91f864680b0c0084eb3f44d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:42.900338 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f1f3cc-aa08-4eca-8d39-3e18675536ad service placement] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:42.901284 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f1f3cc-aa08-4eca-8d39-3e18675536ad service placement] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:42.902414 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f1f3cc-aa08-4eca-8d39-3e18675536ad service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:42.904230 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-48f1f3cc-aa08-4eca-8d39-3e18675536ad service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:16:42.921316 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f1f3cc-aa08-4eca-8d39-3e18675536ad service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:42.924813 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f1f3cc-aa08-4eca-8d39-3e18675536ad service placement] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:42.939497 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 863/1695] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:16:42 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:16:42.940491 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f1f3cc-aa08-4eca-8d39-3e18675536ad service placement] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:42.942307 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-48f1f3cc-aa08-4eca-8d39-3e18675536ad service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:16:42.999517 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 833/1696] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:16:42 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 396 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:16:43.001688 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d2bb5d-de8a-44b7-8e3a-4db312f4aaa5 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:43.005921 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-f2d2bb5d-de8a-44b7-8e3a-4db312f4aaa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:16:43.122549 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f1f3cc-aa08-4eca-8d39-3e18675536ad service placement] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:43.129114 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d2bb5d-de8a-44b7-8e3a-4db312f4aaa5 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:43.136584 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bf8f05-7e97-46c6-b4c6-11234ec758d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:43.137826 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1dbe02-de35-41f8-8ed1-2b9e49d179c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:43.138915 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1dbe02-de35-41f8-8ed1-2b9e49d179c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:43.139473 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1dbe02-de35-41f8-8ed1-2b9e49d179c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:43.139841 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1dbe02-de35-41f8-8ed1-2b9e49d179c5 admin admin] PATH_INFO: `/v3/users/12f9bc9d92d745858d0202adb53c4f16` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:43.140563 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bf8f05-7e97-46c6-b4c6-11234ec758d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:43.141356 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1dbe02-de35-41f8-8ed1-2b9e49d179c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=12f9bc9d92d745858d0202adb53c4f16)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:43.144091 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bf8f05-7e97-46c6-b4c6-11234ec758d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:43.144560 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bf8f05-7e97-46c6-b4c6-11234ec758d9 admin admin] PATH_INFO: `/v3/users/ef7fdbe955364009a891d2b6cbef1db1` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:43.145199 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1dbe02-de35-41f8-8ed1-2b9e49d179c5 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:43.149083 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bf8f05-7e97-46c6-b4c6-11234ec758d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef7fdbe955364009a891d2b6cbef1db1)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:43.151777 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bf8f05-7e97-46c6-b4c6-11234ec758d9 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:43.422621 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-b4bf8f05-7e97-46c6-b4c6-11234ec758d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef7fdbe955364009a891d2b6cbef1db1'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:16:43.430921 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-ea1dbe02-de35-41f8-8ed1-2b9e49d179c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '12f9bc9d92d745858d0202adb53c4f16'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:16:43.431897 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-ea1dbe02-de35-41f8-8ed1-2b9e49d179c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '12f9bc9d92d745858d0202adb53c4f16'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:16:43.439212 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-b4bf8f05-7e97-46c6-b4c6-11234ec758d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef7fdbe955364009a891d2b6cbef1db1'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:16:43.440138 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-b4bf8f05-7e97-46c6-b4c6-11234ec758d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef7fdbe955364009a891d2b6cbef1db1'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:16:43.511511 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-ea1dbe02-de35-41f8-8ed1-2b9e49d179c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '12f9bc9d92d745858d0202adb53c4f16'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:16:43.532268 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-ea1dbe02-de35-41f8-8ed1-2b9e49d179c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '12f9bc9d92d745858d0202adb53c4f16'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:16:43.555717 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-ea1dbe02-de35-41f8-8ed1-2b9e49d179c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '12f9bc9d92d745858d0202adb53c4f16'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:16:43.562480 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 834/1697] 10.222.0.23 () {58 vars in 1223 bytes} [Sat Jun 18 05:16:42 2022] DELETE /identity/v3/users/12f9bc9d92d745858d0202adb53c4f16 => generated 0 bytes in 562 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:16:43.565713 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1dbe02-de35-41f8-8ed1-2b9e49d179c5 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:43.574371 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-ea1dbe02-de35-41f8-8ed1-2b9e49d179c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:16:43.596547 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-b4bf8f05-7e97-46c6-b4c6-11234ec758d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef7fdbe955364009a891d2b6cbef1db1'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:16:43.612620 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-b4bf8f05-7e97-46c6-b4c6-11234ec758d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef7fdbe955364009a891d2b6cbef1db1'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:16:43.621797 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 864/1698] 10.222.0.23 () {58 vars in 1223 bytes} [Sat Jun 18 05:16:42 2022] DELETE /identity/v3/users/ef7fdbe955364009a891d2b6cbef1db1 => generated 0 bytes in 682 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:16:43.622816 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bf8f05-7e97-46c6-b4c6-11234ec758d9 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:43.628915 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-b4bf8f05-7e97-46c6-b4c6-11234ec758d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:16:43.747620 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:16:43.772978 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1dbe02-de35-41f8-8ed1-2b9e49d179c5 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:43.940870 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bf8f05-7e97-46c6-b4c6-11234ec758d9 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:43.958076 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539d4e57-2ca6-4208-bbef-08b8500a1089 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15e7f45ee0744c1d9602187eac8e8b80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:43.959042 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539d4e57-2ca6-4208-bbef-08b8500a1089 service nova] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:43.959494 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539d4e57-2ca6-4208-bbef-08b8500a1089 service nova] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:43.960597 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539d4e57-2ca6-4208-bbef-08b8500a1089 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:43.960963 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a36cfaa-554b-4247-b9d2-894f277e1278 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:43.962014 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a36cfaa-554b-4247-b9d2-894f277e1278 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:43.962400 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a36cfaa-554b-4247-b9d2-894f277e1278 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:43.962773 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a36cfaa-554b-4247-b9d2-894f277e1278 admin admin] PATH_INFO: `/v3/users/f31b4faefebd4d0baeb6840e3166513f` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:43.967735 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a36cfaa-554b-4247-b9d2-894f277e1278 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f31b4faefebd4d0baeb6840e3166513f)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:43.969209 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-539d4e57-2ca6-4208-bbef-08b8500a1089 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:16:43.980343 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a36cfaa-554b-4247-b9d2-894f277e1278 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:44.093337 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:16:44.188898 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539d4e57-2ca6-4208-bbef-08b8500a1089 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:44.190789 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539d4e57-2ca6-4208-bbef-08b8500a1089 service nova] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:44.213360 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 865/1699] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:16:43 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 590 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:16:44.217784 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ab58cb-0e0e-40d8-99ae-01327d73ada9 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:44.219419 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ab58cb-0e0e-40d8-99ae-01327d73ada9 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:44.219682 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ab58cb-0e0e-40d8-99ae-01327d73ada9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:44.264388 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-48ab58cb-0e0e-40d8-99ae-01327d73ada9 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:16:44.264388 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-48ab58cb-0e0e-40d8-99ae-01327d73ada9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:16:44.274995 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 866/1700] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:16:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:16:44.289803 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-8a36cfaa-554b-4247-b9d2-894f277e1278 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f31b4faefebd4d0baeb6840e3166513f'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:16:44.291038 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-8a36cfaa-554b-4247-b9d2-894f277e1278 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f31b4faefebd4d0baeb6840e3166513f'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:16:44.292946 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ab58cb-0e0e-40d8-99ae-01327d73ada9 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:44.298719 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-48ab58cb-0e0e-40d8-99ae-01327d73ada9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:16:44.397116 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-8a36cfaa-554b-4247-b9d2-894f277e1278 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f31b4faefebd4d0baeb6840e3166513f'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:16:44.417779 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-8a36cfaa-554b-4247-b9d2-894f277e1278 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f31b4faefebd4d0baeb6840e3166513f'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:16:44.435234 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-8a36cfaa-554b-4247-b9d2-894f277e1278 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f31b4faefebd4d0baeb6840e3166513f'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:16:44.451361 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 835/1701] 10.222.0.23 () {58 vars in 1223 bytes} [Sat Jun 18 05:16:43 2022] DELETE /identity/v3/users/f31b4faefebd4d0baeb6840e3166513f => generated 0 bytes in 888 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:16:44.452423 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a36cfaa-554b-4247-b9d2-894f277e1278 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:44.455439 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-8a36cfaa-554b-4247-b9d2-894f277e1278 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:16:44.525983 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ab58cb-0e0e-40d8-99ae-01327d73ada9 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:44.610302 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a36cfaa-554b-4247-b9d2-894f277e1278 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:44.626532 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3db7bf-b9ca-499e-9d0f-c8e04a27167b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:44.627493 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3db7bf-b9ca-499e-9d0f-c8e04a27167b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:44.627786 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3db7bf-b9ca-499e-9d0f-c8e04a27167b admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:44.633136 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3db7bf-b9ca-499e-9d0f-c8e04a27167b admin admin] PATH_INFO: `/v3/projects/2ed2f69568d0430ab74917f326a3022f` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:44.633136 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3db7bf-b9ca-499e-9d0f-c8e04a27167b admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ed2f69568d0430ab74917f326a3022f)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:44.634460 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3db7bf-b9ca-499e-9d0f-c8e04a27167b admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:44.696643 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9a788c-fb06-44cc-8724-32524c89607f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c4d7cde8f4ad9aa479bb09e325db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:44.697341 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9a788c-fb06-44cc-8724-32524c89607f service neutron] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:44.697714 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9a788c-fb06-44cc-8724-32524c89607f service neutron] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:44.697995 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9a788c-fb06-44cc-8724-32524c89607f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:44.708779 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-2f9a788c-fb06-44cc-8724-32524c89607f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:16:44.723066 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9a788c-fb06-44cc-8724-32524c89607f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:44.727158 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9a788c-fb06-44cc-8724-32524c89607f service neutron] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:44.755000 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 867/1702] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:16:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 463 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:16:44.807585 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-5b3db7bf-b9ca-499e-9d0f-c8e04a27167b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ed2f69568d0430ab74917f326a3022f', 'request_id': 'req-5b3db7bf-b9ca-499e-9d0f-c8e04a27167b'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:16:44.824165 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-5b3db7bf-b9ca-499e-9d0f-c8e04a27167b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ed2f69568d0430ab74917f326a3022f', 'request_id': 'req-5b3db7bf-b9ca-499e-9d0f-c8e04a27167b'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:16:44.909691 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-5b3db7bf-b9ca-499e-9d0f-c8e04a27167b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ed2f69568d0430ab74917f326a3022f', 'request_id': 'req-5b3db7bf-b9ca-499e-9d0f-c8e04a27167b'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:16:44.934088 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-5b3db7bf-b9ca-499e-9d0f-c8e04a27167b admin admin] The token cache is being invalidate because project 2ed2f69568d0430ab74917f326a3022f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80468) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 18 05:16:44.934765 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-5b3db7bf-b9ca-499e-9d0f-c8e04a27167b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:16:44.936249 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-5b3db7bf-b9ca-499e-9d0f-c8e04a27167b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:16:44.939555 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 836/1703] 10.222.0.23 () {58 vars in 1232 bytes} [Sat Jun 18 05:16:44 2022] DELETE /identity/v3/projects/2ed2f69568d0430ab74917f326a3022f => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:16:44.946774 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a73a81-e6aa-4da1-b24a-0ade38372800 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:44.948671 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a73a81-e6aa-4da1-b24a-0ade38372800 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:44.951367 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a73a81-e6aa-4da1-b24a-0ade38372800 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:44.980030 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-b2a73a81-e6aa-4da1-b24a-0ade38372800 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:16:44.983758 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-b2a73a81-e6aa-4da1-b24a-0ade38372800 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:16:45.161675 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 868/1704] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:16:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:16:45.168795 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460c3f1c-8567-450b-ae2f-08a614226a7e None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:45.169255 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460c3f1c-8567-450b-ae2f-08a614226a7e None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:45.169573 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460c3f1c-8567-450b-ae2f-08a614226a7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:45.213034 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a73a81-e6aa-4da1-b24a-0ade38372800 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:45.214800 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-b2a73a81-e6aa-4da1-b24a-0ade38372800 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:16:45.233010 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-460c3f1c-8567-450b-ae2f-08a614226a7e None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:16:45.234283 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-460c3f1c-8567-450b-ae2f-08a614226a7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:16:45.242754 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a73a81-e6aa-4da1-b24a-0ade38372800 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:45.264334 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 837/1705] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:16:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:16:45.268347 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040b70b5-b1e0-4cef-b9d9-acde1a34045f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:45.268554 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460c3f1c-8567-450b-ae2f-08a614226a7e None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:45.269248 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040b70b5-b1e0-4cef-b9d9-acde1a34045f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:45.269568 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040b70b5-b1e0-4cef-b9d9-acde1a34045f admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:45.269848 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040b70b5-b1e0-4cef-b9d9-acde1a34045f admin admin] PATH_INFO: `/v3/users/f17d7d82e185481da1ddb8e688b2cffe` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:45.272652 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040b70b5-b1e0-4cef-b9d9-acde1a34045f admin admin] RBAC: Authorizing `identity:delete_user(user_id=f17d7d82e185481da1ddb8e688b2cffe)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:45.274953 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040b70b5-b1e0-4cef-b9d9-acde1a34045f admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:45.282694 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460c3f1c-8567-450b-ae2f-08a614226a7e None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:45.306518 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc249328-06d3-4cc2-9f7c-60ecd4c8e987 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:45.307392 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc249328-06d3-4cc2-9f7c-60ecd4c8e987 admin admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:45.307809 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc249328-06d3-4cc2-9f7c-60ecd4c8e987 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:45.309054 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc249328-06d3-4cc2-9f7c-60ecd4c8e987 admin admin] PATH_INFO: `/v3/domains` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:45.309985 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc249328-06d3-4cc2-9f7c-60ecd4c8e987 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 05:16:45.310456 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc249328-06d3-4cc2-9f7c-60ecd4c8e987 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:45.315321 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc249328-06d3-4cc2-9f7c-60ecd4c8e987 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:45.347971 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 838/1706] 10.222.0.23 () {58 vars in 1152 bytes} [Sat Jun 18 05:16:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 05:16:45.351779 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc249328-06d3-4cc2-9f7c-60ecd4c8e987 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:45.376759 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc249328-06d3-4cc2-9f7c-60ecd4c8e987 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:45.383484 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db7840b-2293-4d09-a3c1-5b53c17f36a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:45.385579 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db7840b-2293-4d09-a3c1-5b53c17f36a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:45.385884 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db7840b-2293-4d09-a3c1-5b53c17f36a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:45.386170 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db7840b-2293-4d09-a3c1-5b53c17f36a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:45.387968 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db7840b-2293-4d09-a3c1-5b53c17f36a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:45.391119 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db7840b-2293-4d09-a3c1-5b53c17f36a3 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:45.533776 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 839/1707] 10.222.0.23 () {60 vars in 1152 bytes} [Sat Jun 18 05:16:45 2022] POST /identity/v3/projects => generated 381 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:16:45.541618 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db7840b-2293-4d09-a3c1-5b53c17f36a3 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:45.569080 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:16:45.570198 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-040b70b5-b1e0-4cef-b9d9-acde1a34045f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f17d7d82e185481da1ddb8e688b2cffe'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:16:45.647796 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-040b70b5-b1e0-4cef-b9d9-acde1a34045f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f17d7d82e185481da1ddb8e688b2cffe'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:16:45.649342 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-040b70b5-b1e0-4cef-b9d9-acde1a34045f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f17d7d82e185481da1ddb8e688b2cffe'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:16:45.745984 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-040b70b5-b1e0-4cef-b9d9-acde1a34045f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f17d7d82e185481da1ddb8e688b2cffe'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:16:45.761124 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-040b70b5-b1e0-4cef-b9d9-acde1a34045f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f17d7d82e185481da1ddb8e688b2cffe'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:16:45.769825 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 869/1708] 10.222.0.23 () {58 vars in 1223 bytes} [Sat Jun 18 05:16:45 2022] DELETE /identity/v3/users/f17d7d82e185481da1ddb8e688b2cffe => generated 0 bytes in 557 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 18 05:16:45.784246 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3c1133-ced7-4fd2-8185-4f67c26e278b None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:45.784543 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3c1133-ced7-4fd2-8185-4f67c26e278b None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:45.784832 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3c1133-ced7-4fd2-8185-4f67c26e278b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:45.791518 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db7840b-2293-4d09-a3c1-5b53c17f36a3 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:45.853520 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:16:45.868123 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-6c3c1133-ced7-4fd2-8185-4f67c26e278b None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:16:45.869734 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-6c3c1133-ced7-4fd2-8185-4f67c26e278b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:16:46.003717 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22fd998-2e37-4164-b066-47429c5fbe04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:46.004431 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22fd998-2e37-4164-b066-47429c5fbe04 admin admin] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:46.007360 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22fd998-2e37-4164-b066-47429c5fbe04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:46.007540 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22fd998-2e37-4164-b066-47429c5fbe04 admin admin] PATH_INFO: `/v3/users` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:46.008302 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22fd998-2e37-4164-b066-47429c5fbe04 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:46.019089 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22fd998-2e37-4164-b066-47429c5fbe04 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:46.031541 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 870/1709] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:16:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:16:46.031541 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3c1133-ced7-4fd2-8185-4f67c26e278b None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:46.035266 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-6c3c1133-ced7-4fd2-8185-4f67c26e278b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:16:46.063814 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3c1133-ced7-4fd2-8185-4f67c26e278b None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:46.077695 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf63aaba-feac-43b8-b297-e25927e5d763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:46.080219 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf63aaba-feac-43b8-b297-e25927e5d763 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:46.093958 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf63aaba-feac-43b8-b297-e25927e5d763 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:46.093958 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf63aaba-feac-43b8-b297-e25927e5d763 admin admin] PATH_INFO: `/v3/projects/b97df413b0b140a0aceac04617fb59bd` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:46.093958 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf63aaba-feac-43b8-b297-e25927e5d763 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b97df413b0b140a0aceac04617fb59bd)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:46.095191 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf63aaba-feac-43b8-b297-e25927e5d763 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:46.113443 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 840/1710] 10.222.0.23 () {60 vars in 1143 bytes} [Sat Jun 18 05:16:45 2022] POST /identity/v3/users => generated 291 bytes in 574 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:16:46.114496 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22fd998-2e37-4164-b066-47429c5fbe04 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:46.121602 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-b22fd998-2e37-4164-b066-47429c5fbe04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:16:46.199909 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:16:46.309856 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22fd998-2e37-4164-b066-47429c5fbe04 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:46.315257 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-bf63aaba-feac-43b8-b297-e25927e5d763 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b97df413b0b140a0aceac04617fb59bd', 'request_id': 'req-bf63aaba-feac-43b8-b297-e25927e5d763'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:16:46.327875 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-bf63aaba-feac-43b8-b297-e25927e5d763 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b97df413b0b140a0aceac04617fb59bd', 'request_id': 'req-bf63aaba-feac-43b8-b297-e25927e5d763'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:16:46.338056 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-bf63aaba-feac-43b8-b297-e25927e5d763 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b97df413b0b140a0aceac04617fb59bd', 'request_id': 'req-bf63aaba-feac-43b8-b297-e25927e5d763'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:16:46.401708 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660e65fc-0432-4c4a-bf3c-e449d4dcaf76 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c4d7cde8f4ad9aa479bb09e325db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:46.402844 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660e65fc-0432-4c4a-bf3c-e449d4dcaf76 service neutron] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:46.403318 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660e65fc-0432-4c4a-bf3c-e449d4dcaf76 service neutron] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:46.403738 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660e65fc-0432-4c4a-bf3c-e449d4dcaf76 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:46.447681 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-bf63aaba-feac-43b8-b297-e25927e5d763 admin admin] The token cache is being invalidate because project b97df413b0b140a0aceac04617fb59bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80469) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 18 05:16:46.447971 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-bf63aaba-feac-43b8-b297-e25927e5d763 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:16:46.449498 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-bf63aaba-feac-43b8-b297-e25927e5d763 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:16:46.458259 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 871/1711] 10.222.0.23 () {58 vars in 1232 bytes} [Sat Jun 18 05:16:46 2022] DELETE /identity/v3/projects/b97df413b0b140a0aceac04617fb59bd => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:16:46.463681 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf63aaba-feac-43b8-b297-e25927e5d763 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:46.468590 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-bf63aaba-feac-43b8-b297-e25927e5d763 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:16:46.526315 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660e65fc-0432-4c4a-bf3c-e449d4dcaf76 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:46.529542 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660e65fc-0432-4c4a-bf3c-e449d4dcaf76 service neutron] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:46.558568 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 841/1712] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:16:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:16:46.561836 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0906bd-1bbc-459c-b14d-8a8cf85ea13c None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:46.564656 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0906bd-1bbc-459c-b14d-8a8cf85ea13c None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:46.566996 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0906bd-1bbc-459c-b14d-8a8cf85ea13c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:46.623153 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-8f0906bd-1bbc-459c-b14d-8a8cf85ea13c None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:16:46.627376 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-8f0906bd-1bbc-459c-b14d-8a8cf85ea13c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:16:46.650120 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf63aaba-feac-43b8-b297-e25927e5d763 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:46.653706 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 842/1713] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:16:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:16:46.655503 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0906bd-1bbc-459c-b14d-8a8cf85ea13c None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:46.659950 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-8f0906bd-1bbc-459c-b14d-8a8cf85ea13c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:16:46.669625 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2c5f93-69b1-4acb-9d67-4468db4ceb19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:46.671061 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2c5f93-69b1-4acb-9d67-4468db4ceb19 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:46.672821 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2c5f93-69b1-4acb-9d67-4468db4ceb19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:46.673118 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2c5f93-69b1-4acb-9d67-4468db4ceb19 admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:46.673863 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2c5f93-69b1-4acb-9d67-4468db4ceb19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:46.675488 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2c5f93-69b1-4acb-9d67-4468db4ceb19 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:46.694778 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0906bd-1bbc-459c-b14d-8a8cf85ea13c None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:46.697655 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 872/1714] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:16:46 2022] GET /identity/v3/roles => generated 1420 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:16:46.700584 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae2fb1e-ad75-4a60-ba7b-78d18d358f54 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:46.700961 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae2fb1e-ad75-4a60-ba7b-78d18d358f54 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:46.701478 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae2fb1e-ad75-4a60-ba7b-78d18d358f54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:46.716652 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1fb085-55eb-45c0-a73a-71fff864e333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:46.716652 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1fb085-55eb-45c0-a73a-71fff864e333 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:46.716652 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1fb085-55eb-45c0-a73a-71fff864e333 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:46.716652 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1fb085-55eb-45c0-a73a-71fff864e333 admin admin] PATH_INFO: `/v3/projects/a684f86fb5554337a8ba861ffb29239b` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:46.718266 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1fb085-55eb-45c0-a73a-71fff864e333 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a684f86fb5554337a8ba861ffb29239b)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:46.720308 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1fb085-55eb-45c0-a73a-71fff864e333 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:46.723240 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-bae2fb1e-ad75-4a60-ba7b-78d18d358f54 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:16:46.724479 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-bae2fb1e-ad75-4a60-ba7b-78d18d358f54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:16:46.740541 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 873/1715] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:16:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:16:46.742454 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae2fb1e-ad75-4a60-ba7b-78d18d358f54 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:46.758028 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae2fb1e-ad75-4a60-ba7b-78d18d358f54 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:46.766123 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7edd3f-3fdd-468f-8733-a6dbd67ae47e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:46.767132 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7edd3f-3fdd-468f-8733-a6dbd67ae47e admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:46.767240 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7edd3f-3fdd-468f-8733-a6dbd67ae47e admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:46.767773 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7edd3f-3fdd-468f-8733-a6dbd67ae47e admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:46.768635 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7edd3f-3fdd-468f-8733-a6dbd67ae47e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:46.774049 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7edd3f-3fdd-468f-8733-a6dbd67ae47e admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:46.792517 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 874/1716] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:16:46 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:16:46.804146 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7edd3f-3fdd-468f-8733-a6dbd67ae47e admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:46.842943 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7edd3f-3fdd-468f-8733-a6dbd67ae47e admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:46.864272 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c8f6b1-3104-43b3-ab2a-687633eb0a0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:46.864948 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c8f6b1-3104-43b3-ab2a-687633eb0a0d admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:46.866387 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c8f6b1-3104-43b3-ab2a-687633eb0a0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:46.866387 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c8f6b1-3104-43b3-ab2a-687633eb0a0d admin admin] PATH_INFO: `/v3/domains` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:46.866566 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c8f6b1-3104-43b3-ab2a-687633eb0a0d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 05:16:46.867054 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c8f6b1-3104-43b3-ab2a-687633eb0a0d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:46.868789 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c8f6b1-3104-43b3-ab2a-687633eb0a0d admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:46.886965 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 875/1717] 10.222.0.23 () {58 vars in 1152 bytes} [Sat Jun 18 05:16:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 05:16:46.888373 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c8f6b1-3104-43b3-ab2a-687633eb0a0d admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:46.901153 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c8f6b1-3104-43b3-ab2a-687633eb0a0d admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:46.907633 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3c04c0-436b-4767-88a3-1122409b32cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:46.908586 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3c04c0-436b-4767-88a3-1122409b32cf admin admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:46.911867 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3c04c0-436b-4767-88a3-1122409b32cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:46.911867 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3c04c0-436b-4767-88a3-1122409b32cf admin admin] PATH_INFO: `/v3/projects/45c5494315cd497282ff00dd01b8ed9e/users/80bee726fd9f4d84b2637e9f5e8d3767/roles/cbf5c4c26898413d93e73b46b41046a3` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:46.930811 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3c04c0-436b-4767-88a3-1122409b32cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=45c5494315cd497282ff00dd01b8ed9e, user_id=80bee726fd9f4d84b2637e9f5e8d3767, role_id=cbf5c4c26898413d93e73b46b41046a3)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:46.934726 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3c04c0-436b-4767-88a3-1122409b32cf admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:46.956820 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-da1fb085-55eb-45c0-a73a-71fff864e333 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a684f86fb5554337a8ba861ffb29239b', 'request_id': 'req-da1fb085-55eb-45c0-a73a-71fff864e333'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:16:46.963879 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-da1fb085-55eb-45c0-a73a-71fff864e333 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a684f86fb5554337a8ba861ffb29239b', 'request_id': 'req-da1fb085-55eb-45c0-a73a-71fff864e333'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:16:47.027580 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 876/1718] 10.222.0.23 () {60 vars in 1482 bytes} [Sat Jun 18 05:16:46 2022] PUT /identity/v3/projects/45c5494315cd497282ff00dd01b8ed9e/users/80bee726fd9f4d84b2637e9f5e8d3767/roles/cbf5c4c26898413d93e73b46b41046a3 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:16:47.028425 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3c04c0-436b-4767-88a3-1122409b32cf admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:47.064919 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-da1fb085-55eb-45c0-a73a-71fff864e333 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a684f86fb5554337a8ba861ffb29239b', 'request_id': 'req-da1fb085-55eb-45c0-a73a-71fff864e333'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:16:47.090770 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-da1fb085-55eb-45c0-a73a-71fff864e333 admin admin] The token cache is being invalidate because project a684f86fb5554337a8ba861ffb29239b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80468) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 18 05:16:47.091662 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-da1fb085-55eb-45c0-a73a-71fff864e333 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:16:47.092578 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-da1fb085-55eb-45c0-a73a-71fff864e333 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:16:47.096659 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 843/1719] 10.222.0.23 () {58 vars in 1232 bytes} [Sat Jun 18 05:16:46 2022] DELETE /identity/v3/projects/a684f86fb5554337a8ba861ffb29239b => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:16:47.102711 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08343b6-6e49-494d-a019-543c7e8b07ce None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:47.106392 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08343b6-6e49-494d-a019-543c7e8b07ce None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:47.112817 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08343b6-6e49-494d-a019-543c7e8b07ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:47.177435 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-b08343b6-6e49-494d-a019-543c7e8b07ce None None] MFA Rules not processed for user `80bee726fd9f4d84b2637e9f5e8d3767`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:16:47.181316 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-b08343b6-6e49-494d-a019-543c7e8b07ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:16:47.193449 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3c04c0-436b-4767-88a3-1122409b32cf admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:47.201052 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5119b3f2-e4f2-4d52-be25-25587bebdd5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:47.202041 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5119b3f2-e4f2-4d52-be25-25587bebdd5b admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:47.202607 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5119b3f2-e4f2-4d52-be25-25587bebdd5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:47.203126 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5119b3f2-e4f2-4d52-be25-25587bebdd5b admin admin] PATH_INFO: `/v3/projects` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:47.204029 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5119b3f2-e4f2-4d52-be25-25587bebdd5b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:47.206390 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5119b3f2-e4f2-4d52-be25-25587bebdd5b admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:47.314374 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 844/1720] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:16:47 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:16:47.316261 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8672bb65-dbd0-4141-b687-0b9be8311659 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:47.316806 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8672bb65-dbd0-4141-b687-0b9be8311659 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:47.317485 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8672bb65-dbd0-4141-b687-0b9be8311659 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:47.349068 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-8672bb65-dbd0-4141-b687-0b9be8311659 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:16:47.351501 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-8672bb65-dbd0-4141-b687-0b9be8311659 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:16:47.384823 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 845/1721] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:16:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:16:47.390704 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8672bb65-dbd0-4141-b687-0b9be8311659 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:47.392460 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 877/1722] 10.222.0.23 () {60 vars in 1152 bytes} [Sat Jun 18 05:16:47 2022] POST /identity/v3/projects => generated 381 bytes in 365 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:16:47.392582 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-8672bb65-dbd0-4141-b687-0b9be8311659 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:16:47.401512 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc0d920-dfa0-4b40-8cb4-42cbec962033 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:47.402479 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc0d920-dfa0-4b40-8cb4-42cbec962033 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:47.402960 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc0d920-dfa0-4b40-8cb4-42cbec962033 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:47.433951 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8672bb65-dbd0-4141-b687-0b9be8311659 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:47.442367 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-9bc0d920-dfa0-4b40-8cb4-42cbec962033 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:16:47.447416 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-9bc0d920-dfa0-4b40-8cb4-42cbec962033 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:16:47.579842 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 878/1723] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:16:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:16:47.580909 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc0d920-dfa0-4b40-8cb4-42cbec962033 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:47.586194 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-9bc0d920-dfa0-4b40-8cb4-42cbec962033 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:16:47.588741 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c814c3e6-f749-47f5-b12e-e03ca428ca7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:47.589739 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c814c3e6-f749-47f5-b12e-e03ca428ca7d admin admin] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:47.592057 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c814c3e6-f749-47f5-b12e-e03ca428ca7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:47.592738 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c814c3e6-f749-47f5-b12e-e03ca428ca7d admin admin] PATH_INFO: `/v3/projects` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:47.594151 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c814c3e6-f749-47f5-b12e-e03ca428ca7d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:47.596188 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c814c3e6-f749-47f5-b12e-e03ca428ca7d admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:47.611721 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc0d920-dfa0-4b40-8cb4-42cbec962033 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:47.623928 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7ac85a-2008-4698-ac42-07588ef6b7d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:47.624882 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7ac85a-2008-4698-ac42-07588ef6b7d3 admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:47.630812 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7ac85a-2008-4698-ac42-07588ef6b7d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:47.630812 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7ac85a-2008-4698-ac42-07588ef6b7d3 admin admin] PATH_INFO: `/v3/users` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:47.630812 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7ac85a-2008-4698-ac42-07588ef6b7d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:47.632118 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7ac85a-2008-4698-ac42-07588ef6b7d3 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:47.748990 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 879/1724] 10.222.0.23 () {60 vars in 1143 bytes} [Sat Jun 18 05:16:47 2022] POST /identity/v3/users => generated 291 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:16:47.750993 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13aff36f-eb75-458e-aaa3-8d8620d327ee None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:47.752886 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13aff36f-eb75-458e-aaa3-8d8620d327ee None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:47.753283 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13aff36f-eb75-458e-aaa3-8d8620d327ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:47.774091 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 846/1725] 10.222.0.23 () {60 vars in 1152 bytes} [Sat Jun 18 05:16:47 2022] POST /identity/v3/projects => generated 381 bytes in 385 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:16:47.774091 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c814c3e6-f749-47f5-b12e-e03ca428ca7d admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:47.802645 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-13aff36f-eb75-458e-aaa3-8d8620d327ee None None] MFA Rules not processed for user `d2c8f964ee6a49cda501ed28a9432c8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:16:47.804541 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-13aff36f-eb75-458e-aaa3-8d8620d327ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:16:48.009633 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:16:48.026247 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c814c3e6-f749-47f5-b12e-e03ca428ca7d admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:48.028502 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 880/1726] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:16:47 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:16:48.029522 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13aff36f-eb75-458e-aaa3-8d8620d327ee None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:48.039181 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-13aff36f-eb75-458e-aaa3-8d8620d327ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:16:48.055715 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e321e7f2-adf4-4c21-a089-309215eceedc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:48.056158 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e321e7f2-adf4-4c21-a089-309215eceedc admin admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:48.056445 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e321e7f2-adf4-4c21-a089-309215eceedc admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:48.056722 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e321e7f2-adf4-4c21-a089-309215eceedc admin admin] PATH_INFO: `/v3/domains` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:48.057488 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e321e7f2-adf4-4c21-a089-309215eceedc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 05:16:48.058012 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e321e7f2-adf4-4c21-a089-309215eceedc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:48.059605 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e321e7f2-adf4-4c21-a089-309215eceedc admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:48.078727 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13aff36f-eb75-458e-aaa3-8d8620d327ee None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:48.110783 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943d3083-f738-44b6-b53d-df73e2a3d8d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:48.111662 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943d3083-f738-44b6-b53d-df73e2a3d8d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:48.111952 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943d3083-f738-44b6-b53d-df73e2a3d8d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:48.112229 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943d3083-f738-44b6-b53d-df73e2a3d8d1 admin admin] PATH_INFO: `/v3/projects/d9b8a8dba3d042d797bfb6bf3cef552d` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:48.115378 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943d3083-f738-44b6-b53d-df73e2a3d8d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9b8a8dba3d042d797bfb6bf3cef552d)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:48.124070 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943d3083-f738-44b6-b53d-df73e2a3d8d1 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:48.126665 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 847/1727] 10.222.0.23 () {58 vars in 1152 bytes} [Sat Jun 18 05:16:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 355 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 05:16:48.127645 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e321e7f2-adf4-4c21-a089-309215eceedc admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:48.198612 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e321e7f2-adf4-4c21-a089-309215eceedc admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:48.227617 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1dc138-dac9-42eb-b4ca-b01b1029cb27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:48.230373 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1dc138-dac9-42eb-b4ca-b01b1029cb27 admin admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:48.230680 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1dc138-dac9-42eb-b4ca-b01b1029cb27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:48.230948 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1dc138-dac9-42eb-b4ca-b01b1029cb27 admin admin] PATH_INFO: `/v3/roles` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:48.231694 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1dc138-dac9-42eb-b4ca-b01b1029cb27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:48.233314 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1dc138-dac9-42eb-b4ca-b01b1029cb27 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:48.274365 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 848/1728] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:16:48 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:16:48.274365 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1dc138-dac9-42eb-b4ca-b01b1029cb27 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:48.309519 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1dc138-dac9-42eb-b4ca-b01b1029cb27 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:48.321248 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d177bd-ac75-49ee-a214-b1765d87393e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:48.323488 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d177bd-ac75-49ee-a214-b1765d87393e admin admin] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:48.323488 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d177bd-ac75-49ee-a214-b1765d87393e admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:48.323488 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d177bd-ac75-49ee-a214-b1765d87393e admin admin] PATH_INFO: `/v3/users` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:48.323488 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d177bd-ac75-49ee-a214-b1765d87393e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:48.328062 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d177bd-ac75-49ee-a214-b1765d87393e admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:48.385404 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-943d3083-f738-44b6-b53d-df73e2a3d8d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9b8a8dba3d042d797bfb6bf3cef552d', 'request_id': 'req-943d3083-f738-44b6-b53d-df73e2a3d8d1'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:16:48.426973 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:16:48.432516 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-943d3083-f738-44b6-b53d-df73e2a3d8d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9b8a8dba3d042d797bfb6bf3cef552d', 'request_id': 'req-943d3083-f738-44b6-b53d-df73e2a3d8d1'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:16:48.438810 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 849/1729] 10.222.0.23 () {60 vars in 1143 bytes} [Sat Jun 18 05:16:48 2022] POST /identity/v3/users => generated 297 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:16:48.440025 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d177bd-ac75-49ee-a214-b1765d87393e admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:48.452996 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-943d3083-f738-44b6-b53d-df73e2a3d8d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9b8a8dba3d042d797bfb6bf3cef552d', 'request_id': 'req-943d3083-f738-44b6-b53d-df73e2a3d8d1'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:16:48.466289 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d177bd-ac75-49ee-a214-b1765d87393e admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:48.544708 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-943d3083-f738-44b6-b53d-df73e2a3d8d1 admin admin] The token cache is being invalidate because project d9b8a8dba3d042d797bfb6bf3cef552d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80469) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 18 05:16:48.544996 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-943d3083-f738-44b6-b53d-df73e2a3d8d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:16:48.547102 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-943d3083-f738-44b6-b53d-df73e2a3d8d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:16:48.556529 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 881/1730] 10.222.0.23 () {58 vars in 1232 bytes} [Sat Jun 18 05:16:48 2022] DELETE /identity/v3/projects/d9b8a8dba3d042d797bfb6bf3cef552d => generated 0 bytes in 527 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:16:48.558027 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943d3083-f738-44b6-b53d-df73e2a3d8d1 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:48.560523 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-943d3083-f738-44b6-b53d-df73e2a3d8d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:16:48.666462 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d75e41-2b51-464e-9092-c8df78b9c6c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:48.667399 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d75e41-2b51-464e-9092-c8df78b9c6c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:48.667792 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d75e41-2b51-464e-9092-c8df78b9c6c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:48.668265 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d75e41-2b51-464e-9092-c8df78b9c6c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:48.669110 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d75e41-2b51-464e-9092-c8df78b9c6c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:48.683574 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d75e41-2b51-464e-9092-c8df78b9c6c8 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:48.723856 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943d3083-f738-44b6-b53d-df73e2a3d8d1 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:48.734534 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54870554-1845-4f29-bf19-2151b848657b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:48.735245 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54870554-1845-4f29-bf19-2151b848657b admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:48.735534 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54870554-1845-4f29-bf19-2151b848657b admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:48.735811 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54870554-1845-4f29-bf19-2151b848657b admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:48.736564 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54870554-1845-4f29-bf19-2151b848657b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:48.753968 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54870554-1845-4f29-bf19-2151b848657b admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:48.782750 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 882/1731] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:16:48 2022] GET /identity/v3/roles => generated 1420 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:16:48.783697 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54870554-1845-4f29-bf19-2151b848657b admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:48.790562 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 850/1732] 10.222.0.23 () {60 vars in 1152 bytes} [Sat Jun 18 05:16:48 2022] POST /identity/v3/projects => generated 379 bytes in 351 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:16:48.794611 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836fc02b-219f-4b17-be51-86e66728a5d9 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:48.795475 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-54870554-1845-4f29-bf19-2151b848657b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:16:48.795903 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836fc02b-219f-4b17-be51-86e66728a5d9 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:48.796445 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836fc02b-219f-4b17-be51-86e66728a5d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:48.817152 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54870554-1845-4f29-bf19-2151b848657b admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:48.834960 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-836fc02b-219f-4b17-be51-86e66728a5d9 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:16:48.837731 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-836fc02b-219f-4b17-be51-86e66728a5d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:16:48.944510 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4624844d-15a7-4bd8-8239-29f691c5d55a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:48.945148 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4624844d-15a7-4bd8-8239-29f691c5d55a admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:48.949045 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4624844d-15a7-4bd8-8239-29f691c5d55a admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:48.949045 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4624844d-15a7-4bd8-8239-29f691c5d55a admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:48.949045 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4624844d-15a7-4bd8-8239-29f691c5d55a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:48.949045 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4624844d-15a7-4bd8-8239-29f691c5d55a admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:48.964295 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 883/1733] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:16:48 2022] GET /identity/v3/roles => generated 1420 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:16:48.964295 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4de6467-887d-4a75-a572-ee7f6c7029a4 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:48.964295 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4de6467-887d-4a75-a572-ee7f6c7029a4 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:48.964295 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4de6467-887d-4a75-a572-ee7f6c7029a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:49.009332 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-c4de6467-887d-4a75-a572-ee7f6c7029a4 None None] MFA Rules not processed for user `e817fafd0c1b4bf9944481cf116f1260`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:16:49.010608 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-c4de6467-887d-4a75-a572-ee7f6c7029a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:16:49.041151 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 851/1734] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:16:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:16:49.041811 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836fc02b-219f-4b17-be51-86e66728a5d9 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:49.067920 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836fc02b-219f-4b17-be51-86e66728a5d9 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:49.074557 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6502a57a-6543-4edf-bf1d-b79f8f830837 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:49.075722 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6502a57a-6543-4edf-bf1d-b79f8f830837 admin admin] REQUEST_METHOD: `PUT` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:49.076189 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6502a57a-6543-4edf-bf1d-b79f8f830837 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:49.076570 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6502a57a-6543-4edf-bf1d-b79f8f830837 admin admin] PATH_INFO: `/v3/projects/3dfdbabe16354cd086783a2d6bc63fed/users/aa6ff0b2e3d343c0b2921d3cdb269066/roles/cbf5c4c26898413d93e73b46b41046a3` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:49.115855 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 884/1735] 10.222.0.23 () {58 vars in 1009 bytes} [Sat Jun 18 05:16:48 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:16:49.117115 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4de6467-887d-4a75-a572-ee7f6c7029a4 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:49.118575 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-c4de6467-887d-4a75-a572-ee7f6c7029a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:16:49.129182 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6502a57a-6543-4edf-bf1d-b79f8f830837 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3dfdbabe16354cd086783a2d6bc63fed, user_id=aa6ff0b2e3d343c0b2921d3cdb269066, role_id=cbf5c4c26898413d93e73b46b41046a3)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:49.136937 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6502a57a-6543-4edf-bf1d-b79f8f830837 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:49.147012 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4de6467-887d-4a75-a572-ee7f6c7029a4 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:49.155947 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0e4196-3be4-4821-8f96-02d97adad798 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:49.155947 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0e4196-3be4-4821-8f96-02d97adad798 admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:49.155947 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0e4196-3be4-4821-8f96-02d97adad798 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:49.155947 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0e4196-3be4-4821-8f96-02d97adad798 admin admin] PATH_INFO: `/v3/users` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:49.156546 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0e4196-3be4-4821-8f96-02d97adad798 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:49.159188 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0e4196-3be4-4821-8f96-02d97adad798 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:49.247028 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 852/1736] 10.222.0.23 () {60 vars in 1482 bytes} [Sat Jun 18 05:16:49 2022] PUT /identity/v3/projects/3dfdbabe16354cd086783a2d6bc63fed/users/aa6ff0b2e3d343c0b2921d3cdb269066/roles/cbf5c4c26898413d93e73b46b41046a3 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:16:49.247028 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6502a57a-6543-4edf-bf1d-b79f8f830837 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:49.284147 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 885/1737] 10.222.0.23 () {60 vars in 1143 bytes} [Sat Jun 18 05:16:49 2022] POST /identity/v3/users => generated 290 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:16:49.288041 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e120c7c-f627-46c1-8fb2-7c2fbce0448c None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:49.288489 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e120c7c-f627-46c1-8fb2-7c2fbce0448c None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:49.289214 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e120c7c-f627-46c1-8fb2-7c2fbce0448c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:49.335544 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-9e120c7c-f627-46c1-8fb2-7c2fbce0448c None None] MFA Rules not processed for user `e0f3999a062e45748878a3204e3ea456`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:16:49.339424 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-9e120c7c-f627-46c1-8fb2-7c2fbce0448c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:16:49.394405 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6502a57a-6543-4edf-bf1d-b79f8f830837 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:49.400423 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048bc2b6-7926-426f-a820-48ce95fb7648 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:49.401799 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048bc2b6-7926-426f-a820-48ce95fb7648 admin admin] REQUEST_METHOD: `PUT` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:49.402233 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048bc2b6-7926-426f-a820-48ce95fb7648 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:49.402650 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048bc2b6-7926-426f-a820-48ce95fb7648 admin admin] PATH_INFO: `/v3/projects/35413584bc0e486682d9f439fac5a887/users/4554a3d40e7e458288f88832d83c56b2/roles/21c65f2c51bd4dd283d13507d2714467` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:49.420514 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048bc2b6-7926-426f-a820-48ce95fb7648 admin admin] RBAC: Authorizing `identity:create_grant(project_id=35413584bc0e486682d9f439fac5a887, user_id=4554a3d40e7e458288f88832d83c56b2, role_id=21c65f2c51bd4dd283d13507d2714467)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:49.423690 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048bc2b6-7926-426f-a820-48ce95fb7648 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:49.431390 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:INSERT=3,keystone:SELECT=94,keystone:DELETE=3 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:16:49.500949 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 853/1738] 10.222.0.23 () {60 vars in 1482 bytes} [Sat Jun 18 05:16:49 2022] PUT /identity/v3/projects/35413584bc0e486682d9f439fac5a887/users/4554a3d40e7e458288f88832d83c56b2/roles/21c65f2c51bd4dd283d13507d2714467 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:16:49.502608 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a99fa9-7fe9-4b89-8baa-6b004886ba01 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:49.502905 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a99fa9-7fe9-4b89-8baa-6b004886ba01 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:49.503182 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a99fa9-7fe9-4b89-8baa-6b004886ba01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:49.532882 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 886/1739] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:16:49 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:16:49.535199 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc6cea0-344b-46e5-8dde-8e4836896eac None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:49.535632 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc6cea0-344b-46e5-8dde-8e4836896eac None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:49.536043 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc6cea0-344b-46e5-8dde-8e4836896eac None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:49.538981 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 887/1740] 10.222.0.23 () {54 vars in 912 bytes} [Sat Jun 18 05:16:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 05:16:49.540408 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b60af38-78da-4ac1-8d0d-a40b01114413 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:49.540838 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b60af38-78da-4ac1-8d0d-a40b01114413 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:49.541307 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b60af38-78da-4ac1-8d0d-a40b01114413 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:49.551392 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-45a99fa9-7fe9-4b89-8baa-6b004886ba01 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:16:49.553505 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-45a99fa9-7fe9-4b89-8baa-6b004886ba01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:16:49.588858 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-2b60af38-78da-4ac1-8d0d-a40b01114413 None None] MFA Rules not processed for user `aa6ff0b2e3d343c0b2921d3cdb269066`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:16:49.590104 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-2b60af38-78da-4ac1-8d0d-a40b01114413 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:16:49.661641 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 854/1741] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:16:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:16:49.663666 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a99fa9-7fe9-4b89-8baa-6b004886ba01 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:49.677173 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a99fa9-7fe9-4b89-8baa-6b004886ba01 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:49.682006 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=90,keystone:INSERT=4,keystone:DELETE=6 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:16:49.683859 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 888/1742] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:16:49 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:16:49.684900 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b60af38-78da-4ac1-8d0d-a40b01114413 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:49.686291 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc26bb6f-d3f3-473c-b668-4be9abcc3b83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:49.686595 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-2b60af38-78da-4ac1-8d0d-a40b01114413 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:16:49.687146 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc26bb6f-d3f3-473c-b668-4be9abcc3b83 admin admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:49.687511 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc26bb6f-d3f3-473c-b668-4be9abcc3b83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:49.687859 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc26bb6f-d3f3-473c-b668-4be9abcc3b83 admin admin] PATH_INFO: `/v3/roles` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:49.688831 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc26bb6f-d3f3-473c-b668-4be9abcc3b83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:49.690727 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc26bb6f-d3f3-473c-b668-4be9abcc3b83 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:49.708046 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 855/1743] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:16:49 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:16:49.708046 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc26bb6f-d3f3-473c-b668-4be9abcc3b83 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:49.721011 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc26bb6f-d3f3-473c-b668-4be9abcc3b83 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:49.727058 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b1987c-1dce-4f2d-98c1-b1da5202978d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:49.727781 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b1987c-1dce-4f2d-98c1-b1da5202978d admin admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:49.728071 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b1987c-1dce-4f2d-98c1-b1da5202978d admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:49.728399 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b1987c-1dce-4f2d-98c1-b1da5202978d admin admin] PATH_INFO: `/v3/roles` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:49.729131 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b1987c-1dce-4f2d-98c1-b1da5202978d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:49.731778 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b1987c-1dce-4f2d-98c1-b1da5202978d admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:49.744072 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 856/1744] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:16:49 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:16:49.745699 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edebfaf4-211a-4a01-82cf-6171daeb593e None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:49.746114 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edebfaf4-211a-4a01-82cf-6171daeb593e None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:49.747107 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edebfaf4-211a-4a01-82cf-6171daeb593e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:49.795146 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b60af38-78da-4ac1-8d0d-a40b01114413 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:49.800284 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-edebfaf4-211a-4a01-82cf-6171daeb593e None None] MFA Rules not processed for user `e78c4d7cde8f4ad9aa479bb09e325db3`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:16:49.802978 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-edebfaf4-211a-4a01-82cf-6171daeb593e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:16:49.806761 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b21d47-3584-42e4-9983-c21bc72333aa service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16b49e866398400580ae48aa5a96fc04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:49.807363 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b21d47-3584-42e4-9983-c21bc72333aa service swift] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:49.807655 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b21d47-3584-42e4-9983-c21bc72333aa service swift] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:49.807933 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b21d47-3584-42e4-9983-c21bc72333aa service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:49.883962 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b21d47-3584-42e4-9983-c21bc72333aa service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:49.885810 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b21d47-3584-42e4-9983-c21bc72333aa service swift] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:49.906350 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 889/1745] 10.222.0.23 () {60 vars in 1384 bytes} [Sat Jun 18 05:16:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 18 05:16:49.911897 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 857/1746] 10.222.0.23 () {58 vars in 1009 bytes} [Sat Jun 18 05:16:49 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:16:49.915369 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b21d47-3584-42e4-9983-c21bc72333aa service swift] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:49.923923 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edebfaf4-211a-4a01-82cf-6171daeb593e None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:49.927573 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b21d47-3584-42e4-9983-c21bc72333aa service swift] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:49.941010 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982686b2-a61d-4e61-977a-c7ba35fa384f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:49.945817 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982686b2-a61d-4e61-977a-c7ba35fa384f admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:49.945817 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982686b2-a61d-4e61-977a-c7ba35fa384f admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:49.945817 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982686b2-a61d-4e61-977a-c7ba35fa384f admin admin] PATH_INFO: `/v3/domains` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:49.945817 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982686b2-a61d-4e61-977a-c7ba35fa384f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 05:16:49.945817 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982686b2-a61d-4e61-977a-c7ba35fa384f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:49.947728 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982686b2-a61d-4e61-977a-c7ba35fa384f admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:49.962966 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edebfaf4-211a-4a01-82cf-6171daeb593e None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:49.968046 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 890/1747] 10.222.0.23 () {58 vars in 1152 bytes} [Sat Jun 18 05:16:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 05:16:49.968218 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62ad851-5de4-477f-9b4a-31306d85b43d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:49.968737 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982686b2-a61d-4e61-977a-c7ba35fa384f admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:49.968894 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62ad851-5de4-477f-9b4a-31306d85b43d admin admin] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:49.969184 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62ad851-5de4-477f-9b4a-31306d85b43d admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:49.969482 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62ad851-5de4-477f-9b4a-31306d85b43d admin admin] PATH_INFO: `/v3/projects` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:49.971884 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62ad851-5de4-477f-9b4a-31306d85b43d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:49.975870 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62ad851-5de4-477f-9b4a-31306d85b43d admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:49.992456 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982686b2-a61d-4e61-977a-c7ba35fa384f admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:49.999567 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1634983d-c734-443b-8654-371ddcff018d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:50.000606 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1634983d-c734-443b-8654-371ddcff018d admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:50.000994 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1634983d-c734-443b-8654-371ddcff018d admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:50.001481 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1634983d-c734-443b-8654-371ddcff018d admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:50.002749 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1634983d-c734-443b-8654-371ddcff018d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:50.015297 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1634983d-c734-443b-8654-371ddcff018d admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:50.043570 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 891/1748] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:16:49 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:16:50.043570 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1634983d-c734-443b-8654-371ddcff018d admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:50.059629 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1634983d-c734-443b-8654-371ddcff018d admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:50.068742 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830e857c-cecc-4785-88d4-61c67af37a69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:50.071426 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830e857c-cecc-4785-88d4-61c67af37a69 admin admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:50.071820 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830e857c-cecc-4785-88d4-61c67af37a69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:50.072199 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830e857c-cecc-4785-88d4-61c67af37a69 admin admin] PATH_INFO: `/v3/projects/35413584bc0e486682d9f439fac5a887/users/4554a3d40e7e458288f88832d83c56b2/roles/21c65f2c51bd4dd283d13507d2714467` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:50.078275 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830e857c-cecc-4785-88d4-61c67af37a69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=35413584bc0e486682d9f439fac5a887, user_id=4554a3d40e7e458288f88832d83c56b2, role_id=21c65f2c51bd4dd283d13507d2714467)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:50.084047 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830e857c-cecc-4785-88d4-61c67af37a69 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:50.120516 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 892/1749] 10.222.0.23 () {60 vars in 1482 bytes} [Sat Jun 18 05:16:50 2022] PUT /identity/v3/projects/35413584bc0e486682d9f439fac5a887/users/4554a3d40e7e458288f88832d83c56b2/roles/21c65f2c51bd4dd283d13507d2714467 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:16:50.122377 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830e857c-cecc-4785-88d4-61c67af37a69 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:50.183254 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 858/1750] 10.222.0.23 () {60 vars in 1152 bytes} [Sat Jun 18 05:16:49 2022] POST /identity/v3/projects => generated 381 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:16:50.188826 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62ad851-5de4-477f-9b4a-31306d85b43d admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:50.344787 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830e857c-cecc-4785-88d4-61c67af37a69 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:50.450765 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62ad851-5de4-477f-9b4a-31306d85b43d admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:50.467519 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:16:50.488387 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cfb47f-da1e-4c97-aa52-53816493ed39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:50.489310 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cfb47f-da1e-4c97-aa52-53816493ed39 admin admin] REQUEST_METHOD: `PUT` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:50.489689 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cfb47f-da1e-4c97-aa52-53816493ed39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:50.489945 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cfb47f-da1e-4c97-aa52-53816493ed39 admin admin] PATH_INFO: `/v3/projects/95cca8e47eb34d698fabe95982e89dbd/users/512ba151c38d4ef2a6c2b04604a1c12a/roles/cbf5c4c26898413d93e73b46b41046a3` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:50.513544 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:16:50.529025 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cfb47f-da1e-4c97-aa52-53816493ed39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=95cca8e47eb34d698fabe95982e89dbd, user_id=512ba151c38d4ef2a6c2b04604a1c12a, role_id=cbf5c4c26898413d93e73b46b41046a3)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:50.531998 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cfb47f-da1e-4c97-aa52-53816493ed39 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:50.544677 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd973f79-eaa6-41fe-8017-2add6162a213 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:50.545338 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd973f79-eaa6-41fe-8017-2add6162a213 admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:50.545655 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd973f79-eaa6-41fe-8017-2add6162a213 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:50.545955 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd973f79-eaa6-41fe-8017-2add6162a213 admin admin] PATH_INFO: `/v3/projects` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:50.549749 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd973f79-eaa6-41fe-8017-2add6162a213 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:50.557599 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd973f79-eaa6-41fe-8017-2add6162a213 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:50.629292 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 859/1751] 10.222.0.23 () {60 vars in 1482 bytes} [Sat Jun 18 05:16:50 2022] PUT /identity/v3/projects/95cca8e47eb34d698fabe95982e89dbd/users/512ba151c38d4ef2a6c2b04604a1c12a/roles/cbf5c4c26898413d93e73b46b41046a3 => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:16:50.632119 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cfb47f-da1e-4c97-aa52-53816493ed39 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:50.634673 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-11cfb47f-da1e-4c97-aa52-53816493ed39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:16:50.702171 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 893/1752] 10.222.0.23 () {60 vars in 1152 bytes} [Sat Jun 18 05:16:50 2022] POST /identity/v3/projects => generated 373 bytes in 581 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:16:50.703676 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d4df0c-a75a-44e9-89fa-45be8a85af34 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:50.704022 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d4df0c-a75a-44e9-89fa-45be8a85af34 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:50.704305 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d4df0c-a75a-44e9-89fa-45be8a85af34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:50.762570 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cfb47f-da1e-4c97-aa52-53816493ed39 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:50.813441 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-85d4df0c-a75a-44e9-89fa-45be8a85af34 None None] MFA Rules not processed for user `4554a3d40e7e458288f88832d83c56b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:16:50.814918 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-85d4df0c-a75a-44e9-89fa-45be8a85af34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:16:50.919803 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1282fe4-ee77-4813-bb29-a8936f8c18a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c4d7cde8f4ad9aa479bb09e325db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:50.920507 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1282fe4-ee77-4813-bb29-a8936f8c18a9 service neutron] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:50.920763 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1282fe4-ee77-4813-bb29-a8936f8c18a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:50.921044 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1282fe4-ee77-4813-bb29-a8936f8c18a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:50.947951 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1282fe4-ee77-4813-bb29-a8936f8c18a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:50.950050 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1282fe4-ee77-4813-bb29-a8936f8c18a9 service neutron] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:50.963315 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 860/1753] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:16:50 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:16:50.964248 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1282fe4-ee77-4813-bb29-a8936f8c18a9 service neutron] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:50.991071 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:16:51.024422 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 894/1754] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:16:50 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 320 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:16:51.025942 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8bc882-098b-4984-ac19-3975e1ab1494 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:51.026243 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8bc882-098b-4984-ac19-3975e1ab1494 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:51.026928 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8bc882-098b-4984-ac19-3975e1ab1494 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:51.074530 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1282fe4-ee77-4813-bb29-a8936f8c18a9 service neutron] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:51.092410 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e3ad4a-7d44-4606-9748-772ca3c923ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:51.093363 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e3ad4a-7d44-4606-9748-772ca3c923ee admin admin] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:51.093662 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e3ad4a-7d44-4606-9748-772ca3c923ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:51.094012 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e3ad4a-7d44-4606-9748-772ca3c923ee admin admin] PATH_INFO: `/v3/users` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:51.098706 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-8b8bc882-098b-4984-ac19-3975e1ab1494 None None] MFA Rules not processed for user `512ba151c38d4ef2a6c2b04604a1c12a`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:16:51.100163 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e3ad4a-7d44-4606-9748-772ca3c923ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:51.103503 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-8b8bc882-098b-4984-ac19-3975e1ab1494 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:16:51.105807 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:16:51.107707 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e3ad4a-7d44-4606-9748-772ca3c923ee admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:51.226138 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 895/1755] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:16:51 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:16:51.226138 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7db97a8-71bb-4770-880c-fe508a838539 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:51.226138 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7db97a8-71bb-4770-880c-fe508a838539 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:51.226138 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7db97a8-71bb-4770-880c-fe508a838539 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:51.279541 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-d7db97a8-71bb-4770-880c-fe508a838539 None None] MFA Rules not processed for user `e78c4d7cde8f4ad9aa479bb09e325db3`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:16:51.280828 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-d7db97a8-71bb-4770-880c-fe508a838539 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:16:51.339993 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 896/1756] 10.222.0.15 () {58 vars in 1015 bytes} [Sat Jun 18 05:16:51 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:16:51.339993 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7db97a8-71bb-4770-880c-fe508a838539 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:51.340283 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 861/1757] 10.222.0.23 () {60 vars in 1143 bytes} [Sat Jun 18 05:16:50 2022] POST /identity/v3/users => generated 297 bytes in 356 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:16:51.340283 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e3ad4a-7d44-4606-9748-772ca3c923ee admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:51.340283 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-05e3ad4a-7d44-4606-9748-772ca3c923ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:16:51.340698 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7db97a8-71bb-4770-880c-fe508a838539 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:51.342212 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfc5d02-8780-4d7f-8081-ffa8faeb15c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:51.343520 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfc5d02-8780-4d7f-8081-ffa8faeb15c1 admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:51.344094 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfc5d02-8780-4d7f-8081-ffa8faeb15c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:51.344834 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfc5d02-8780-4d7f-8081-ffa8faeb15c1 admin admin] PATH_INFO: `/v3/users` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:51.345985 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfc5d02-8780-4d7f-8081-ffa8faeb15c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:51.348817 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfc5d02-8780-4d7f-8081-ffa8faeb15c1 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:51.413295 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e3ad4a-7d44-4606-9748-772ca3c923ee admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:51.417334 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0f699a-590b-4a3d-af0a-1d1b33544fed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9de0428abc8400aa338aba4f7c34c94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:51.418113 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0f699a-590b-4a3d-af0a-1d1b33544fed service cinder] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:51.418557 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0f699a-590b-4a3d-af0a-1d1b33544fed service cinder] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:51.418947 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0f699a-590b-4a3d-af0a-1d1b33544fed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:51.431417 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0f699a-590b-4a3d-af0a-1d1b33544fed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:51.433397 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0f699a-590b-4a3d-af0a-1d1b33544fed service cinder] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:51.445176 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 862/1758] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:16:51 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:16:51.446230 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0f699a-590b-4a3d-af0a-1d1b33544fed service cinder] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:51.469892 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0f699a-590b-4a3d-af0a-1d1b33544fed service cinder] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:51.476554 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2003ae-86d3-450d-80e1-5962d216cd13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:51.477601 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2003ae-86d3-450d-80e1-5962d216cd13 admin admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:51.478137 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2003ae-86d3-450d-80e1-5962d216cd13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:51.478583 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2003ae-86d3-450d-80e1-5962d216cd13 admin admin] PATH_INFO: `/v3/roles` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:51.479652 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2003ae-86d3-450d-80e1-5962d216cd13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:51.481635 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2003ae-86d3-450d-80e1-5962d216cd13 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:51.499850 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 863/1759] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:16:51 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:16:51.501203 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2003ae-86d3-450d-80e1-5962d216cd13 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:51.514761 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2003ae-86d3-450d-80e1-5962d216cd13 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:51.536702 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44204262-628f-4291-b83c-24ea9114caba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c4d7cde8f4ad9aa479bb09e325db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:51.537369 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44204262-628f-4291-b83c-24ea9114caba service neutron] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:51.537713 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44204262-628f-4291-b83c-24ea9114caba service neutron] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:51.538059 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44204262-628f-4291-b83c-24ea9114caba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:51.558249 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44204262-628f-4291-b83c-24ea9114caba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:51.560512 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44204262-628f-4291-b83c-24ea9114caba service neutron] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:51.583783 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 864/1760] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:16:51 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:16:51.583783 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44204262-628f-4291-b83c-24ea9114caba service neutron] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:51.583783 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-44204262-628f-4291-b83c-24ea9114caba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:16:51.611313 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44204262-628f-4291-b83c-24ea9114caba service neutron] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:51.618342 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 897/1761] 10.222.0.23 () {60 vars in 1143 bytes} [Sat Jun 18 05:16:51 2022] POST /identity/v3/users => generated 287 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:16:51.620225 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfc5d02-8780-4d7f-8081-ffa8faeb15c1 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:51.622808 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0542da-4590-45ca-b293-cf4e3eeb586d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9de0428abc8400aa338aba4f7c34c94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:51.623506 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0542da-4590-45ca-b293-cf4e3eeb586d service cinder] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:51.624205 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0542da-4590-45ca-b293-cf4e3eeb586d service cinder] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:51.624653 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0542da-4590-45ca-b293-cf4e3eeb586d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:51.642755 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0542da-4590-45ca-b293-cf4e3eeb586d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:51.647173 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0542da-4590-45ca-b293-cf4e3eeb586d service cinder] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:51.672564 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 865/1762] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:16:51 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:16:51.674901 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0542da-4590-45ca-b293-cf4e3eeb586d service cinder] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:51.687612 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfc5d02-8780-4d7f-8081-ffa8faeb15c1 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:51.694041 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:16:51.704283 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72279b65-5136-4e25-84e0-72aa334493d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:51.710180 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72279b65-5136-4e25-84e0-72aa334493d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:51.710180 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72279b65-5136-4e25-84e0-72aa334493d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:51.710180 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72279b65-5136-4e25-84e0-72aa334493d0 admin admin] PATH_INFO: `/v3/projects/f6485bcce7fb40139f08f61755d773f8/users/baae48a07d7e417486294c655d88b5c0/roles/21c65f2c51bd4dd283d13507d2714467` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:51.727279 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0542da-4590-45ca-b293-cf4e3eeb586d service cinder] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:51.760841 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07da05e-b22c-4873-91ce-1889cb42a82c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:51.767336 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07da05e-b22c-4873-91ce-1889cb42a82c admin admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:51.768005 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07da05e-b22c-4873-91ce-1889cb42a82c admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:51.768005 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07da05e-b22c-4873-91ce-1889cb42a82c admin admin] PATH_INFO: `/v3/roles` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:51.768883 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07da05e-b22c-4873-91ce-1889cb42a82c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:51.781508 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07da05e-b22c-4873-91ce-1889cb42a82c admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:51.793647 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72279b65-5136-4e25-84e0-72aa334493d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6485bcce7fb40139f08f61755d773f8, user_id=baae48a07d7e417486294c655d88b5c0, role_id=21c65f2c51bd4dd283d13507d2714467)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:51.814629 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72279b65-5136-4e25-84e0-72aa334493d0 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:51.913963 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 866/1763] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:16:51 2022] GET /identity/v3/roles => generated 1420 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:16:51.913963 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07da05e-b22c-4873-91ce-1889cb42a82c admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:51.934855 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 898/1764] 10.222.0.23 () {60 vars in 1482 bytes} [Sat Jun 18 05:16:51 2022] PUT /identity/v3/projects/f6485bcce7fb40139f08f61755d773f8/users/baae48a07d7e417486294c655d88b5c0/roles/21c65f2c51bd4dd283d13507d2714467 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:16:51.938861 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72279b65-5136-4e25-84e0-72aa334493d0 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:51.946433 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:16:51.952607 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07da05e-b22c-4873-91ce-1889cb42a82c admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:52.092489 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72279b65-5136-4e25-84e0-72aa334493d0 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:52.107822 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c73fba7-d2fc-407c-bd89-d68d213000a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:52.110717 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e671c331-10a7-48c4-97e8-08b5484da790 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:52.112771 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e671c331-10a7-48c4-97e8-08b5484da790 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:52.114110 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e671c331-10a7-48c4-97e8-08b5484da790 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:52.117485 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e671c331-10a7-48c4-97e8-08b5484da790 admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:52.117485 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e671c331-10a7-48c4-97e8-08b5484da790 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:52.117782 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c73fba7-d2fc-407c-bd89-d68d213000a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:52.117782 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c73fba7-d2fc-407c-bd89-d68d213000a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:52.117782 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c73fba7-d2fc-407c-bd89-d68d213000a0 admin admin] PATH_INFO: `/v3/users/512ba151c38d4ef2a6c2b04604a1c12a` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:52.122166 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e671c331-10a7-48c4-97e8-08b5484da790 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:52.126865 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c73fba7-d2fc-407c-bd89-d68d213000a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=512ba151c38d4ef2a6c2b04604a1c12a)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:52.132139 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c73fba7-d2fc-407c-bd89-d68d213000a0 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:52.186259 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 899/1765] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:16:51 2022] GET /identity/v3/roles => generated 1420 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:16:52.187383 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e671c331-10a7-48c4-97e8-08b5484da790 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:52.234962 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e671c331-10a7-48c4-97e8-08b5484da790 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:52.244700 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58125b3b-5bd6-4234-bb5e-2b71d986b48a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:52.247143 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58125b3b-5bd6-4234-bb5e-2b71d986b48a admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:52.247495 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58125b3b-5bd6-4234-bb5e-2b71d986b48a admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:52.247827 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58125b3b-5bd6-4234-bb5e-2b71d986b48a admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:52.249910 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58125b3b-5bd6-4234-bb5e-2b71d986b48a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:52.254582 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58125b3b-5bd6-4234-bb5e-2b71d986b48a admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:52.308585 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 900/1766] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:16:52 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:16:52.309815 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58125b3b-5bd6-4234-bb5e-2b71d986b48a admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:52.329297 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58125b3b-5bd6-4234-bb5e-2b71d986b48a admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:52.337234 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300ee306-91a7-4d83-97e5-2af5386d4120 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:52.338420 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300ee306-91a7-4d83-97e5-2af5386d4120 admin admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:52.338849 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300ee306-91a7-4d83-97e5-2af5386d4120 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:52.339367 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300ee306-91a7-4d83-97e5-2af5386d4120 admin admin] PATH_INFO: `/v3/projects/62c17eddb9ae4b6698b3706b1a3c8d01/users/760e1aeeff814c2b9aa9023604c3347f/roles/cbf5c4c26898413d93e73b46b41046a3` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:52.379582 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300ee306-91a7-4d83-97e5-2af5386d4120 admin admin] RBAC: Authorizing `identity:create_grant(project_id=62c17eddb9ae4b6698b3706b1a3c8d01, user_id=760e1aeeff814c2b9aa9023604c3347f, role_id=cbf5c4c26898413d93e73b46b41046a3)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:52.383799 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300ee306-91a7-4d83-97e5-2af5386d4120 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:52.420892 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-3c73fba7-d2fc-407c-bd89-d68d213000a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '512ba151c38d4ef2a6c2b04604a1c12a'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:16:52.422451 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-3c73fba7-d2fc-407c-bd89-d68d213000a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '512ba151c38d4ef2a6c2b04604a1c12a'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:16:52.448891 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:16:52.528481 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 901/1767] 10.222.0.23 () {60 vars in 1482 bytes} [Sat Jun 18 05:16:52 2022] PUT /identity/v3/projects/62c17eddb9ae4b6698b3706b1a3c8d01/users/760e1aeeff814c2b9aa9023604c3347f/roles/cbf5c4c26898413d93e73b46b41046a3 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:16:52.530057 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300ee306-91a7-4d83-97e5-2af5386d4120 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:52.532607 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-300ee306-91a7-4d83-97e5-2af5386d4120 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:16:52.534432 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-3c73fba7-d2fc-407c-bd89-d68d213000a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '512ba151c38d4ef2a6c2b04604a1c12a'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:16:52.555342 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-3c73fba7-d2fc-407c-bd89-d68d213000a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '512ba151c38d4ef2a6c2b04604a1c12a'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:16:52.563254 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-3c73fba7-d2fc-407c-bd89-d68d213000a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '512ba151c38d4ef2a6c2b04604a1c12a'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:16:52.574240 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 867/1768] 10.222.0.23 () {58 vars in 1223 bytes} [Sat Jun 18 05:16:51 2022] DELETE /identity/v3/users/512ba151c38d4ef2a6c2b04604a1c12a => generated 0 bytes in 664 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:16:52.575751 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bc33ca-708a-4261-8ee1-8d1f1ca155b9 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:52.576034 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bc33ca-708a-4261-8ee1-8d1f1ca155b9 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:52.576329 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bc33ca-708a-4261-8ee1-8d1f1ca155b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:52.619911 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-e6bc33ca-708a-4261-8ee1-8d1f1ca155b9 None None] MFA Rules not processed for user `760e1aeeff814c2b9aa9023604c3347f`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:16:52.620959 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-e6bc33ca-708a-4261-8ee1-8d1f1ca155b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:16:52.727682 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 868/1769] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:16:52 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:16:52.729761 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b563be-aa00-44a8-9266-770e399a685c None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:52.730224 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b563be-aa00-44a8-9266-770e399a685c None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:52.730649 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b563be-aa00-44a8-9266-770e399a685c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:52.756330 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300ee306-91a7-4d83-97e5-2af5386d4120 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:52.783899 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-55b563be-aa00-44a8-9266-770e399a685c None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:16:52.788376 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-55b563be-aa00-44a8-9266-770e399a685c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:16:52.885571 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:16:52.982815 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34a2828-91a5-47ed-a917-998ddea8bbf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:52.983730 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34a2828-91a5-47ed-a917-998ddea8bbf4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:52.984014 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34a2828-91a5-47ed-a917-998ddea8bbf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:52.984286 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34a2828-91a5-47ed-a917-998ddea8bbf4 admin admin] PATH_INFO: `/v3/projects/f6485bcce7fb40139f08f61755d773f8/users/baae48a07d7e417486294c655d88b5c0/roles/21c65f2c51bd4dd283d13507d2714467` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:52.992295 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34a2828-91a5-47ed-a917-998ddea8bbf4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6485bcce7fb40139f08f61755d773f8, user_id=baae48a07d7e417486294c655d88b5c0, role_id=21c65f2c51bd4dd283d13507d2714467)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:52.995295 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34a2828-91a5-47ed-a917-998ddea8bbf4 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:53.032834 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 902/1770] 10.222.0.23 () {60 vars in 1482 bytes} [Sat Jun 18 05:16:52 2022] PUT /identity/v3/projects/f6485bcce7fb40139f08f61755d773f8/users/baae48a07d7e417486294c655d88b5c0/roles/21c65f2c51bd4dd283d13507d2714467 => generated 0 bytes in 494 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:16:53.033715 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34a2828-91a5-47ed-a917-998ddea8bbf4 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:53.037781 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-b34a2828-91a5-47ed-a917-998ddea8bbf4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:16:53.083812 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 869/1771] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:16:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 355 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:16:53.091256 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8145e92d-b77f-4f83-80fb-91782fb9be81 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:53.091755 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8145e92d-b77f-4f83-80fb-91782fb9be81 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:53.091891 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8145e92d-b77f-4f83-80fb-91782fb9be81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:53.180567 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-8145e92d-b77f-4f83-80fb-91782fb9be81 None None] MFA Rules not processed for user `baae48a07d7e417486294c655d88b5c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:16:53.185041 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-8145e92d-b77f-4f83-80fb-91782fb9be81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:16:53.276483 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:16:53.289416 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34a2828-91a5-47ed-a917-998ddea8bbf4 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:53.307637 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b378e116-cacc-4498-9d5d-cefc29bd7cc9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15e7f45ee0744c1d9602187eac8e8b80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:53.308336 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b378e116-cacc-4498-9d5d-cefc29bd7cc9 service nova] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:53.308990 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b378e116-cacc-4498-9d5d-cefc29bd7cc9 service nova] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:53.308990 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b378e116-cacc-4498-9d5d-cefc29bd7cc9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:53.427361 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:16:53.454241 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b378e116-cacc-4498-9d5d-cefc29bd7cc9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:53.457252 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 870/1772] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:16:53 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 372 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:16:53.458646 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8145e92d-b77f-4f83-80fb-91782fb9be81 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:53.465144 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b378e116-cacc-4498-9d5d-cefc29bd7cc9 service nova] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:53.465271 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-8145e92d-b77f-4f83-80fb-91782fb9be81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:16:53.510086 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 903/1773] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:16:53 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 478 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:16:53.518448 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b378e116-cacc-4498-9d5d-cefc29bd7cc9 service nova] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:53.534499 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-b378e116-cacc-4498-9d5d-cefc29bd7cc9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:16:53.663456 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8145e92d-b77f-4f83-80fb-91782fb9be81 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:53.674517 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad0b77d-5c33-4dbb-982c-ebd48fbeaf23 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c4d7cde8f4ad9aa479bb09e325db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:53.675551 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad0b77d-5c33-4dbb-982c-ebd48fbeaf23 service neutron] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:53.675983 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad0b77d-5c33-4dbb-982c-ebd48fbeaf23 service neutron] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:53.676589 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad0b77d-5c33-4dbb-982c-ebd48fbeaf23 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:53.798979 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b378e116-cacc-4498-9d5d-cefc29bd7cc9 service nova] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:53.802548 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13daf453-4099-48fb-82ba-f742d6e28725 tempest-VolumeQuotaClassesTest-1164682443 tempest-VolumeQuotaClassesTest-1164682443-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4554a3d40e7e458288f88832d83c56b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35413584bc0e486682d9f439fac5a887', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:53.803056 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad0b77d-5c33-4dbb-982c-ebd48fbeaf23 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:53.806182 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13daf453-4099-48fb-82ba-f742d6e28725 tempest-VolumeQuotaClassesTest-1164682443 tempest-VolumeQuotaClassesTest-1164682443-project-admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:53.808800 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad0b77d-5c33-4dbb-982c-ebd48fbeaf23 service neutron] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:53.822395 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:16:53.824624 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13daf453-4099-48fb-82ba-f742d6e28725 tempest-VolumeQuotaClassesTest-1164682443 tempest-VolumeQuotaClassesTest-1164682443-project-admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:53.824923 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13daf453-4099-48fb-82ba-f742d6e28725 tempest-VolumeQuotaClassesTest-1164682443 tempest-VolumeQuotaClassesTest-1164682443-project-admin] PATH_INFO: `/v3/domains` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:53.827438 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13daf453-4099-48fb-82ba-f742d6e28725 tempest-VolumeQuotaClassesTest-1164682443 tempest-VolumeQuotaClassesTest-1164682443-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 05:16:53.828357 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13daf453-4099-48fb-82ba-f742d6e28725 tempest-VolumeQuotaClassesTest-1164682443 tempest-VolumeQuotaClassesTest-1164682443-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:53.833554 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13daf453-4099-48fb-82ba-f742d6e28725 tempest-VolumeQuotaClassesTest-1164682443 tempest-VolumeQuotaClassesTest-1164682443-project-admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:53.840789 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 871/1774] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:16:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:16:53.844414 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad0b77d-5c33-4dbb-982c-ebd48fbeaf23 service neutron] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:53.848563 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-8ad0b77d-5c33-4dbb-982c-ebd48fbeaf23 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:16:53.901035 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 904/1775] 10.222.0.23 () {58 vars in 1152 bytes} [Sat Jun 18 05:16:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 390 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 05:16:53.904484 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62587372-b229-4551-bc48-1d188be73881 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:53.905541 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62587372-b229-4551-bc48-1d188be73881 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:53.909566 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62587372-b229-4551-bc48-1d188be73881 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:53.909566 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 905/1776] 10.222.0.23 () {54 vars in 921 bytes} [Sat Jun 18 05:16:53 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 05:16:53.909566 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62587372-b229-4551-bc48-1d188be73881 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:53.976992 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62587372-b229-4551-bc48-1d188be73881 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:53.986853 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41170414-ad00-43dc-a492-13442c25f551 tempest-VolumeQuotaClassesTest-1164682443 tempest-VolumeQuotaClassesTest-1164682443-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4554a3d40e7e458288f88832d83c56b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35413584bc0e486682d9f439fac5a887', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:53.990428 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41170414-ad00-43dc-a492-13442c25f551 tempest-VolumeQuotaClassesTest-1164682443 tempest-VolumeQuotaClassesTest-1164682443-project-admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:53.990428 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41170414-ad00-43dc-a492-13442c25f551 tempest-VolumeQuotaClassesTest-1164682443 tempest-VolumeQuotaClassesTest-1164682443-project-admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:53.990428 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41170414-ad00-43dc-a492-13442c25f551 tempest-VolumeQuotaClassesTest-1164682443 tempest-VolumeQuotaClassesTest-1164682443-project-admin] PATH_INFO: `/v3/projects` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:53.990428 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41170414-ad00-43dc-a492-13442c25f551 tempest-VolumeQuotaClassesTest-1164682443 tempest-VolumeQuotaClassesTest-1164682443-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:54.002526 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41170414-ad00-43dc-a492-13442c25f551 tempest-VolumeQuotaClassesTest-1164682443 tempest-VolumeQuotaClassesTest-1164682443-project-admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:54.077055 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad0b77d-5c33-4dbb-982c-ebd48fbeaf23 service neutron] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:54.092801 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95727c18-ed11-4a75-a6c6-23388c83b4c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9de0428abc8400aa338aba4f7c34c94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:54.093626 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95727c18-ed11-4a75-a6c6-23388c83b4c0 service cinder] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:54.093936 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95727c18-ed11-4a75-a6c6-23388c83b4c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:54.094264 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95727c18-ed11-4a75-a6c6-23388c83b4c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:54.126076 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95727c18-ed11-4a75-a6c6-23388c83b4c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:54.136288 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95727c18-ed11-4a75-a6c6-23388c83b4c0 service cinder] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:54.186528 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 906/1777] 10.222.0.23 () {60 vars in 1152 bytes} [Sat Jun 18 05:16:53 2022] POST /identity/v3/projects => generated 354 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:16:54.186528 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948993b8-e25d-4a25-a01b-c70edf3f6477 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:54.188443 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948993b8-e25d-4a25-a01b-c70edf3f6477 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:54.188910 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948993b8-e25d-4a25-a01b-c70edf3f6477 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:54.195980 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 872/1778] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:16:53 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:16:54.201726 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95727c18-ed11-4a75-a6c6-23388c83b4c0 service cinder] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:54.252039 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95727c18-ed11-4a75-a6c6-23388c83b4c0 service cinder] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:54.280913 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:16:54.281642 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-948993b8-e25d-4a25-a01b-c70edf3f6477 None None] MFA Rules not processed for user `15e7f45ee0744c1d9602187eac8e8b80`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:16:54.289509 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-948993b8-e25d-4a25-a01b-c70edf3f6477 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:16:54.490704 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4706551a-81bd-4eb2-9bec-f31fc450307a tempest-VolumeQuotaClassesTest-1164682443 tempest-VolumeQuotaClassesTest-1164682443-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4554a3d40e7e458288f88832d83c56b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35413584bc0e486682d9f439fac5a887', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:54.491928 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4706551a-81bd-4eb2-9bec-f31fc450307a tempest-VolumeQuotaClassesTest-1164682443 tempest-VolumeQuotaClassesTest-1164682443-project-admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:54.492245 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4706551a-81bd-4eb2-9bec-f31fc450307a tempest-VolumeQuotaClassesTest-1164682443 tempest-VolumeQuotaClassesTest-1164682443-project-admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:54.492611 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4706551a-81bd-4eb2-9bec-f31fc450307a tempest-VolumeQuotaClassesTest-1164682443 tempest-VolumeQuotaClassesTest-1164682443-project-admin] PATH_INFO: `/v3/domains` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:54.493758 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4706551a-81bd-4eb2-9bec-f31fc450307a tempest-VolumeQuotaClassesTest-1164682443 tempest-VolumeQuotaClassesTest-1164682443-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 05:16:54.499192 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4706551a-81bd-4eb2-9bec-f31fc450307a tempest-VolumeQuotaClassesTest-1164682443 tempest-VolumeQuotaClassesTest-1164682443-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:54.499192 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4706551a-81bd-4eb2-9bec-f31fc450307a tempest-VolumeQuotaClassesTest-1164682443 tempest-VolumeQuotaClassesTest-1164682443-project-admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:54.532298 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 873/1779] 10.222.0.23 () {58 vars in 1152 bytes} [Sat Jun 18 05:16:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 05:16:54.534029 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4706551a-81bd-4eb2-9bec-f31fc450307a tempest-VolumeQuotaClassesTest-1164682443 tempest-VolumeQuotaClassesTest-1164682443-project-admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:54.550005 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-4706551a-81bd-4eb2-9bec-f31fc450307a tempest-VolumeQuotaClassesTest-1164682443 tempest-VolumeQuotaClassesTest-1164682443-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:16:54.577919 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 907/1780] 10.222.0.23 () {58 vars in 1018 bytes} [Sat Jun 18 05:16:54 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 393 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:16:54.578872 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948993b8-e25d-4a25-a01b-c70edf3f6477 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:54.606920 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:16:54.736654 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4706551a-81bd-4eb2-9bec-f31fc450307a tempest-VolumeQuotaClassesTest-1164682443 tempest-VolumeQuotaClassesTest-1164682443-project-admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:54.744848 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948993b8-e25d-4a25-a01b-c70edf3f6477 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:54.748861 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b890c0-07e5-40e0-8d0f-40edb0efd38b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9de0428abc8400aa338aba4f7c34c94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:54.749607 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b890c0-07e5-40e0-8d0f-40edb0efd38b service cinder] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:54.749893 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b890c0-07e5-40e0-8d0f-40edb0efd38b service cinder] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:54.750172 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b890c0-07e5-40e0-8d0f-40edb0efd38b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:54.753123 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78191452-8694-4ef9-96e4-82d0c1e32fbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:54.755082 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78191452-8694-4ef9-96e4-82d0c1e32fbe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:54.755208 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-f3b890c0-07e5-40e0-8d0f-40edb0efd38b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:16:54.755536 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78191452-8694-4ef9-96e4-82d0c1e32fbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:54.755861 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78191452-8694-4ef9-96e4-82d0c1e32fbe admin admin] PATH_INFO: `/v3/users/760e1aeeff814c2b9aa9023604c3347f` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:54.759638 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78191452-8694-4ef9-96e4-82d0c1e32fbe admin admin] RBAC: Authorizing `identity:delete_user(user_id=760e1aeeff814c2b9aa9023604c3347f)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:54.761309 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78191452-8694-4ef9-96e4-82d0c1e32fbe admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:54.914603 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b890c0-07e5-40e0-8d0f-40edb0efd38b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:54.916539 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b890c0-07e5-40e0-8d0f-40edb0efd38b service cinder] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:54.968654 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 908/1781] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:16:54 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:16:54.968654 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b890c0-07e5-40e0-8d0f-40edb0efd38b service cinder] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:54.987981 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b890c0-07e5-40e0-8d0f-40edb0efd38b service cinder] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:54.997617 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5716a4-c79b-4644-99b4-374be3ed77cd tempest-VolumeQuotaClassesTest-1164682443 tempest-VolumeQuotaClassesTest-1164682443-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4554a3d40e7e458288f88832d83c56b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35413584bc0e486682d9f439fac5a887', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:54.998883 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5716a4-c79b-4644-99b4-374be3ed77cd tempest-VolumeQuotaClassesTest-1164682443 tempest-VolumeQuotaClassesTest-1164682443-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:54.999437 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5716a4-c79b-4644-99b4-374be3ed77cd tempest-VolumeQuotaClassesTest-1164682443 tempest-VolumeQuotaClassesTest-1164682443-project-admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:54.999829 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5716a4-c79b-4644-99b4-374be3ed77cd tempest-VolumeQuotaClassesTest-1164682443 tempest-VolumeQuotaClassesTest-1164682443-project-admin] PATH_INFO: `/v3/projects/9b15d4d1932c4f1e9930a3521a8ab996` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:55.010436 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5716a4-c79b-4644-99b4-374be3ed77cd tempest-VolumeQuotaClassesTest-1164682443 tempest-VolumeQuotaClassesTest-1164682443-project-admin] RBAC: Authorizing `identity:delete_project(project_id=9b15d4d1932c4f1e9930a3521a8ab996)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:55.014339 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5716a4-c79b-4644-99b4-374be3ed77cd tempest-VolumeQuotaClassesTest-1164682443 tempest-VolumeQuotaClassesTest-1164682443-project-admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:55.098322 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-78191452-8694-4ef9-96e4-82d0c1e32fbe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '760e1aeeff814c2b9aa9023604c3347f'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:16:55.099815 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-78191452-8694-4ef9-96e4-82d0c1e32fbe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '760e1aeeff814c2b9aa9023604c3347f'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:16:55.204474 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-5f5716a4-c79b-4644-99b4-374be3ed77cd tempest-VolumeQuotaClassesTest-1164682443 tempest-VolumeQuotaClassesTest-1164682443-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b15d4d1932c4f1e9930a3521a8ab996', 'request_id': 'req-5f5716a4-c79b-4644-99b4-374be3ed77cd'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:16:55.229428 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-78191452-8694-4ef9-96e4-82d0c1e32fbe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '760e1aeeff814c2b9aa9023604c3347f'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:16:55.237134 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-5f5716a4-c79b-4644-99b4-374be3ed77cd tempest-VolumeQuotaClassesTest-1164682443 tempest-VolumeQuotaClassesTest-1164682443-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b15d4d1932c4f1e9930a3521a8ab996', 'request_id': 'req-5f5716a4-c79b-4644-99b4-374be3ed77cd'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:16:55.251272 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-5f5716a4-c79b-4644-99b4-374be3ed77cd tempest-VolumeQuotaClassesTest-1164682443 tempest-VolumeQuotaClassesTest-1164682443-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b15d4d1932c4f1e9930a3521a8ab996', 'request_id': 'req-5f5716a4-c79b-4644-99b4-374be3ed77cd'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:16:55.253031 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-78191452-8694-4ef9-96e4-82d0c1e32fbe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '760e1aeeff814c2b9aa9023604c3347f'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:16:55.270822 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-78191452-8694-4ef9-96e4-82d0c1e32fbe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '760e1aeeff814c2b9aa9023604c3347f'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:16:55.288997 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 874/1782] 10.222.0.23 () {58 vars in 1223 bytes} [Sat Jun 18 05:16:54 2022] DELETE /identity/v3/users/760e1aeeff814c2b9aa9023604c3347f => generated 0 bytes in 756 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:16:55.290170 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78191452-8694-4ef9-96e4-82d0c1e32fbe admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:55.300132 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-78191452-8694-4ef9-96e4-82d0c1e32fbe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:16:55.357593 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:16:55.387212 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-5f5716a4-c79b-4644-99b4-374be3ed77cd tempest-VolumeQuotaClassesTest-1164682443 tempest-VolumeQuotaClassesTest-1164682443-project-admin] The token cache is being invalidate because project 9b15d4d1932c4f1e9930a3521a8ab996 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80469) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 18 05:16:55.388129 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-5f5716a4-c79b-4644-99b4-374be3ed77cd tempest-VolumeQuotaClassesTest-1164682443 tempest-VolumeQuotaClassesTest-1164682443-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:16:55.389199 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-5f5716a4-c79b-4644-99b4-374be3ed77cd tempest-VolumeQuotaClassesTest-1164682443 tempest-VolumeQuotaClassesTest-1164682443-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:16:55.403379 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 909/1783] 10.222.0.23 () {58 vars in 1232 bytes} [Sat Jun 18 05:16:54 2022] DELETE /identity/v3/projects/9b15d4d1932c4f1e9930a3521a8ab996 => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:16:55.406184 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5716a4-c79b-4644-99b4-374be3ed77cd tempest-VolumeQuotaClassesTest-1164682443 tempest-VolumeQuotaClassesTest-1164682443-project-admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:55.409131 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-5f5716a4-c79b-4644-99b4-374be3ed77cd tempest-VolumeQuotaClassesTest-1164682443 tempest-VolumeQuotaClassesTest-1164682443-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:16:55.493173 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:16:55.518443 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78191452-8694-4ef9-96e4-82d0c1e32fbe admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:55.527150 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c2d1a2-5d5d-49fe-8ab1-871a47b8458e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15e7f45ee0744c1d9602187eac8e8b80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:55.528054 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c2d1a2-5d5d-49fe-8ab1-871a47b8458e service nova] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:55.544862 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c2d1a2-5d5d-49fe-8ab1-871a47b8458e service nova] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:55.545272 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c2d1a2-5d5d-49fe-8ab1-871a47b8458e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:55.552250 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-21c2d1a2-5d5d-49fe-8ab1-871a47b8458e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:16:55.555571 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5716a4-c79b-4644-99b4-374be3ed77cd tempest-VolumeQuotaClassesTest-1164682443 tempest-VolumeQuotaClassesTest-1164682443-project-admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:55.591181 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c2d1a2-5d5d-49fe-8ab1-871a47b8458e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:55.598321 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c2d1a2-5d5d-49fe-8ab1-871a47b8458e service nova] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:55.624162 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 875/1784] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:16:55 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:16:55.626457 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36070899-a4be-45d9-ba46-efd23244c711 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:55.627563 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36070899-a4be-45d9-ba46-efd23244c711 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:55.628076 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36070899-a4be-45d9-ba46-efd23244c711 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:55.651944 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-36070899-a4be-45d9-ba46-efd23244c711 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:16:55.653756 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-36070899-a4be-45d9-ba46-efd23244c711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:16:55.707842 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fd1218-f147-4915-9878-42facf54911d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:55.709562 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fd1218-f147-4915-9878-42facf54911d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:55.709866 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fd1218-f147-4915-9878-42facf54911d admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:55.710439 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fd1218-f147-4915-9878-42facf54911d admin admin] PATH_INFO: `/v3/projects/95cca8e47eb34d698fabe95982e89dbd` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:55.712264 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fd1218-f147-4915-9878-42facf54911d admin admin] RBAC: Authorizing `identity:delete_project(project_id=95cca8e47eb34d698fabe95982e89dbd)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:55.716341 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fd1218-f147-4915-9878-42facf54911d admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:55.792103 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 876/1785] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:16:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:16:55.803105 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36070899-a4be-45d9-ba46-efd23244c711 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:55.806793 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-36070899-a4be-45d9-ba46-efd23244c711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:16:55.903079 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-d5fd1218-f147-4915-9878-42facf54911d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '95cca8e47eb34d698fabe95982e89dbd', 'request_id': 'req-d5fd1218-f147-4915-9878-42facf54911d'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:16:55.909973 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-d5fd1218-f147-4915-9878-42facf54911d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '95cca8e47eb34d698fabe95982e89dbd', 'request_id': 'req-d5fd1218-f147-4915-9878-42facf54911d'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:16:55.916013 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36070899-a4be-45d9-ba46-efd23244c711 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:55.924766 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-d5fd1218-f147-4915-9878-42facf54911d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '95cca8e47eb34d698fabe95982e89dbd', 'request_id': 'req-d5fd1218-f147-4915-9878-42facf54911d'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:16:56.013740 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427211c5-0c33-4e22-b8f3-86ff8808938a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c4d7cde8f4ad9aa479bb09e325db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:56.017723 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427211c5-0c33-4e22-b8f3-86ff8808938a service neutron] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:56.017723 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427211c5-0c33-4e22-b8f3-86ff8808938a service neutron] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:56.017723 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427211c5-0c33-4e22-b8f3-86ff8808938a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:56.049644 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:16:56.055424 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-d5fd1218-f147-4915-9878-42facf54911d admin admin] The token cache is being invalidate because project 95cca8e47eb34d698fabe95982e89dbd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80469) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 18 05:16:56.055850 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-d5fd1218-f147-4915-9878-42facf54911d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:16:56.056975 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-d5fd1218-f147-4915-9878-42facf54911d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:16:56.061739 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 910/1786] 10.222.0.23 () {58 vars in 1232 bytes} [Sat Jun 18 05:16:55 2022] DELETE /identity/v3/projects/95cca8e47eb34d698fabe95982e89dbd => generated 0 bytes in 654 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:16:56.065726 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2e0229-6cc8-42c1-b46d-81445605c431 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:56.066179 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2e0229-6cc8-42c1-b46d-81445605c431 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:56.066536 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2e0229-6cc8-42c1-b46d-81445605c431 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:56.108466 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-2f2e0229-6cc8-42c1-b46d-81445605c431 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:16:56.111033 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-2f2e0229-6cc8-42c1-b46d-81445605c431 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:16:56.179739 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427211c5-0c33-4e22-b8f3-86ff8808938a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:56.181812 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427211c5-0c33-4e22-b8f3-86ff8808938a service neutron] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:56.200677 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 877/1787] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:16:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:16:56.310802 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 911/1788] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:16:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:16:56.321615 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbc8c69-71d4-417e-bcab-3fcf17c96015 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:56.321956 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbc8c69-71d4-417e-bcab-3fcf17c96015 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:56.323989 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbc8c69-71d4-417e-bcab-3fcf17c96015 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:56.379950 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-8dbc8c69-71d4-417e-bcab-3fcf17c96015 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:16:56.381237 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-8dbc8c69-71d4-417e-bcab-3fcf17c96015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:16:56.400772 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 878/1789] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:16:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:16:56.405469 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2e0229-6cc8-42c1-b46d-81445605c431 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:56.432494 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2e0229-6cc8-42c1-b46d-81445605c431 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:56.444101 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707f511d-ee6a-49b5-8885-ae3fca97ecab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:56.444835 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707f511d-ee6a-49b5-8885-ae3fca97ecab admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:56.445126 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707f511d-ee6a-49b5-8885-ae3fca97ecab admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:56.448036 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbc8c69-71d4-417e-bcab-3fcf17c96015 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:56.451510 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-8dbc8c69-71d4-417e-bcab-3fcf17c96015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:16:56.458487 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707f511d-ee6a-49b5-8885-ae3fca97ecab admin admin] PATH_INFO: `/v3/domains` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:56.459409 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707f511d-ee6a-49b5-8885-ae3fca97ecab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 05:16:56.459920 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707f511d-ee6a-49b5-8885-ae3fca97ecab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:56.461753 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707f511d-ee6a-49b5-8885-ae3fca97ecab admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:56.468809 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbc8c69-71d4-417e-bcab-3fcf17c96015 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:56.478002 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8ea193-79ca-45aa-bd1e-a726526b3b6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:56.479334 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8ea193-79ca-45aa-bd1e-a726526b3b6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:56.479768 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8ea193-79ca-45aa-bd1e-a726526b3b6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:56.480194 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8ea193-79ca-45aa-bd1e-a726526b3b6d admin admin] PATH_INFO: `/v3/users/80bee726fd9f4d84b2637e9f5e8d3767` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:56.481741 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8ea193-79ca-45aa-bd1e-a726526b3b6d admin admin] RBAC: Authorizing `identity:delete_user(user_id=80bee726fd9f4d84b2637e9f5e8d3767)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:56.490746 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8ea193-79ca-45aa-bd1e-a726526b3b6d admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:56.491024 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 912/1790] 10.222.0.23 () {58 vars in 1152 bytes} [Sat Jun 18 05:16:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 05:16:56.491024 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707f511d-ee6a-49b5-8885-ae3fca97ecab admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:56.500189 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707f511d-ee6a-49b5-8885-ae3fca97ecab admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:56.505594 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4b6bb1-f799-4ddf-8874-2958d83261b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:56.506257 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4b6bb1-f799-4ddf-8874-2958d83261b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:56.507532 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4b6bb1-f799-4ddf-8874-2958d83261b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:56.508004 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4b6bb1-f799-4ddf-8874-2958d83261b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:56.508713 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4b6bb1-f799-4ddf-8874-2958d83261b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:56.510710 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4b6bb1-f799-4ddf-8874-2958d83261b7 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:56.515694 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:16:56.652191 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 913/1791] 10.222.0.23 () {60 vars in 1152 bytes} [Sat Jun 18 05:16:56 2022] POST /identity/v3/projects => generated 375 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 18 05:16:56.655202 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4b6bb1-f799-4ddf-8874-2958d83261b7 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:56.747408 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-ed8ea193-79ca-45aa-bd1e-a726526b3b6d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '80bee726fd9f4d84b2637e9f5e8d3767'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:16:56.748183 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-ed8ea193-79ca-45aa-bd1e-a726526b3b6d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '80bee726fd9f4d84b2637e9f5e8d3767'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:16:56.836293 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-ed8ea193-79ca-45aa-bd1e-a726526b3b6d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '80bee726fd9f4d84b2637e9f5e8d3767'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:16:56.861054 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-ed8ea193-79ca-45aa-bd1e-a726526b3b6d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '80bee726fd9f4d84b2637e9f5e8d3767'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:16:56.877400 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-ed8ea193-79ca-45aa-bd1e-a726526b3b6d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '80bee726fd9f4d84b2637e9f5e8d3767'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:16:56.890179 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 879/1792] 10.222.0.23 () {58 vars in 1223 bytes} [Sat Jun 18 05:16:56 2022] DELETE /identity/v3/users/80bee726fd9f4d84b2637e9f5e8d3767 => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:16:56.891048 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8ea193-79ca-45aa-bd1e-a726526b3b6d admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:56.895280 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-ed8ea193-79ca-45aa-bd1e-a726526b3b6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:16:56.971789 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4b6bb1-f799-4ddf-8874-2958d83261b7 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:57.041525 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8ea193-79ca-45aa-bd1e-a726526b3b6d admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:57.057394 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011105b1-0491-43c9-b26f-135b1ea94a65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:57.058936 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011105b1-0491-43c9-b26f-135b1ea94a65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:57.059415 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011105b1-0491-43c9-b26f-135b1ea94a65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:57.059885 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011105b1-0491-43c9-b26f-135b1ea94a65 admin admin] PATH_INFO: `/v3/projects/62c17eddb9ae4b6698b3706b1a3c8d01` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:57.061619 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011105b1-0491-43c9-b26f-135b1ea94a65 admin admin] RBAC: Authorizing `identity:delete_project(project_id=62c17eddb9ae4b6698b3706b1a3c8d01)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:57.064274 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011105b1-0491-43c9-b26f-135b1ea94a65 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:57.082633 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c375b89-95b1-43b6-aec1-2d3bc9bc6169 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:57.083910 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c375b89-95b1-43b6-aec1-2d3bc9bc6169 admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:57.084204 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c375b89-95b1-43b6-aec1-2d3bc9bc6169 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:57.084918 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c375b89-95b1-43b6-aec1-2d3bc9bc6169 admin admin] PATH_INFO: `/v3/users` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:57.085657 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c375b89-95b1-43b6-aec1-2d3bc9bc6169 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:57.087619 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c375b89-95b1-43b6-aec1-2d3bc9bc6169 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:57.101092 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:16:57.209998 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 914/1793] 10.222.0.23 () {60 vars in 1143 bytes} [Sat Jun 18 05:16:56 2022] POST /identity/v3/users => generated 288 bytes in 556 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:16:57.209998 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96930a7f-a487-468f-a43b-73b2d7c1e0a7 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:57.209998 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96930a7f-a487-468f-a43b-73b2d7c1e0a7 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:57.209998 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96930a7f-a487-468f-a43b-73b2d7c1e0a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:57.247914 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-011105b1-0491-43c9-b26f-135b1ea94a65 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '62c17eddb9ae4b6698b3706b1a3c8d01', 'request_id': 'req-011105b1-0491-43c9-b26f-135b1ea94a65'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:16:57.260060 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-011105b1-0491-43c9-b26f-135b1ea94a65 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '62c17eddb9ae4b6698b3706b1a3c8d01', 'request_id': 'req-011105b1-0491-43c9-b26f-135b1ea94a65'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:16:57.268777 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-96930a7f-a487-468f-a43b-73b2d7c1e0a7 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:16:57.271882 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-96930a7f-a487-468f-a43b-73b2d7c1e0a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:16:57.351286 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-011105b1-0491-43c9-b26f-135b1ea94a65 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '62c17eddb9ae4b6698b3706b1a3c8d01', 'request_id': 'req-011105b1-0491-43c9-b26f-135b1ea94a65'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:16:57.374881 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-011105b1-0491-43c9-b26f-135b1ea94a65 admin admin] The token cache is being invalidate because project 62c17eddb9ae4b6698b3706b1a3c8d01 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80468) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 18 05:16:57.377446 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-011105b1-0491-43c9-b26f-135b1ea94a65 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:16:57.380676 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-011105b1-0491-43c9-b26f-135b1ea94a65 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:16:57.395075 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 880/1794] 10.222.0.23 () {58 vars in 1232 bytes} [Sat Jun 18 05:16:56 2022] DELETE /identity/v3/projects/62c17eddb9ae4b6698b3706b1a3c8d01 => generated 0 bytes in 504 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:16:57.400110 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011105b1-0491-43c9-b26f-135b1ea94a65 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:57.410585 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-011105b1-0491-43c9-b26f-135b1ea94a65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:16:57.510856 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 915/1795] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:16:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:16:57.515924 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b78c1c8-8622-43d1-a07f-93311895cbb4 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:57.516220 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b78c1c8-8622-43d1-a07f-93311895cbb4 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:57.516507 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b78c1c8-8622-43d1-a07f-93311895cbb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:57.520213 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011105b1-0491-43c9-b26f-135b1ea94a65 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:57.529545 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a84663f-e757-4963-8688-ac4ac4f275f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:57.529545 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a84663f-e757-4963-8688-ac4ac4f275f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:57.529545 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a84663f-e757-4963-8688-ac4ac4f275f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:57.529545 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a84663f-e757-4963-8688-ac4ac4f275f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:57.529916 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a84663f-e757-4963-8688-ac4ac4f275f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:57.531387 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a84663f-e757-4963-8688-ac4ac4f275f1 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:57.547524 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 881/1796] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:16:57 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:16:57.551103 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a84663f-e757-4963-8688-ac4ac4f275f1 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:57.555461 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-2a84663f-e757-4963-8688-ac4ac4f275f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:16:57.594527 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-9b78c1c8-8622-43d1-a07f-93311895cbb4 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:16:57.595844 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-9b78c1c8-8622-43d1-a07f-93311895cbb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:16:57.626887 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 916/1797] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:16:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:16:57.632806 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b78c1c8-8622-43d1-a07f-93311895cbb4 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:57.643223 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a84663f-e757-4963-8688-ac4ac4f275f1 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:57.644050 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b78c1c8-8622-43d1-a07f-93311895cbb4 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:57.648101 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b683b849-b575-48cf-b2d7-3c4411577739 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c4d7cde8f4ad9aa479bb09e325db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:57.649027 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b683b849-b575-48cf-b2d7-3c4411577739 service neutron] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:57.649363 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b683b849-b575-48cf-b2d7-3c4411577739 service neutron] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:57.649677 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b683b849-b575-48cf-b2d7-3c4411577739 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:57.649989 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b498a0c2-a0ac-4c10-b8aa-3723e29352e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:57.651028 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b498a0c2-a0ac-4c10-b8aa-3723e29352e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:57.651464 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b498a0c2-a0ac-4c10-b8aa-3723e29352e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:57.651826 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b498a0c2-a0ac-4c10-b8aa-3723e29352e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:57.652658 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b498a0c2-a0ac-4c10-b8aa-3723e29352e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:57.654465 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b498a0c2-a0ac-4c10-b8aa-3723e29352e3 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:57.660526 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b683b849-b575-48cf-b2d7-3c4411577739 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:57.662172 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b683b849-b575-48cf-b2d7-3c4411577739 service neutron] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:57.681599 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 882/1798] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:16:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:16:57.683110 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f9f6a8-6920-493c-a75d-2c5caf17ebcb None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:57.683405 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f9f6a8-6920-493c-a75d-2c5caf17ebcb None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:57.683692 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f9f6a8-6920-493c-a75d-2c5caf17ebcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:57.702633 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 917/1799] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:16:57 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:16:57.705956 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b498a0c2-a0ac-4c10-b8aa-3723e29352e3 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:57.719918 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-a3f9f6a8-6920-493c-a75d-2c5caf17ebcb None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:16:57.720480 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:16:57.721343 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-a3f9f6a8-6920-493c-a75d-2c5caf17ebcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:16:57.730165 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b498a0c2-a0ac-4c10-b8aa-3723e29352e3 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:57.730800 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 883/1800] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:16:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:16:57.735745 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f9f6a8-6920-493c-a75d-2c5caf17ebcb None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:57.742598 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0872b3f-d019-4493-bd84-ac926c2415fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:57.747197 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0872b3f-d019-4493-bd84-ac926c2415fd admin admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:57.747197 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0872b3f-d019-4493-bd84-ac926c2415fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:57.747197 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0872b3f-d019-4493-bd84-ac926c2415fd admin admin] PATH_INFO: `/v3/projects/5dc0a53b17c54845883befbac80f3a7a/users/dfab4a1c0ff2430f9f234f790593b47c/roles/cbf5c4c26898413d93e73b46b41046a3` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:57.748718 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f9f6a8-6920-493c-a75d-2c5caf17ebcb None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:57.754942 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35a2dc4-9942-47de-b61a-379fa9adf9ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:57.755776 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35a2dc4-9942-47de-b61a-379fa9adf9ea admin admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:57.756085 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35a2dc4-9942-47de-b61a-379fa9adf9ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:57.756351 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35a2dc4-9942-47de-b61a-379fa9adf9ea admin admin] PATH_INFO: `/v3/domains` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:57.758065 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35a2dc4-9942-47de-b61a-379fa9adf9ea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 05:16:57.758416 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35a2dc4-9942-47de-b61a-379fa9adf9ea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:57.760642 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35a2dc4-9942-47de-b61a-379fa9adf9ea admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:57.771144 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0872b3f-d019-4493-bd84-ac926c2415fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=5dc0a53b17c54845883befbac80f3a7a, user_id=dfab4a1c0ff2430f9f234f790593b47c, role_id=cbf5c4c26898413d93e73b46b41046a3)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:57.775251 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0872b3f-d019-4493-bd84-ac926c2415fd admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:57.804945 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 884/1801] 10.222.0.23 () {58 vars in 1152 bytes} [Sat Jun 18 05:16:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 05:16:57.807307 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35a2dc4-9942-47de-b61a-379fa9adf9ea admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:57.818971 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35a2dc4-9942-47de-b61a-379fa9adf9ea admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:57.823890 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c65ca3a-0e22-46d6-9952-81120238d3c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:57.824560 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c65ca3a-0e22-46d6-9952-81120238d3c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:57.825030 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c65ca3a-0e22-46d6-9952-81120238d3c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:57.825313 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c65ca3a-0e22-46d6-9952-81120238d3c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:57.826127 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c65ca3a-0e22-46d6-9952-81120238d3c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:57.828080 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c65ca3a-0e22-46d6-9952-81120238d3c8 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:57.880096 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 918/1802] 10.222.0.23 () {60 vars in 1482 bytes} [Sat Jun 18 05:16:57 2022] PUT /identity/v3/projects/5dc0a53b17c54845883befbac80f3a7a/users/dfab4a1c0ff2430f9f234f790593b47c/roles/cbf5c4c26898413d93e73b46b41046a3 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:16:57.895630 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c906a6-195f-42af-8ade-5c70fc9e8a84 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:57.895921 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c906a6-195f-42af-8ade-5c70fc9e8a84 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:57.896207 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c906a6-195f-42af-8ade-5c70fc9e8a84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:57.947214 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 885/1803] 10.222.0.23 () {60 vars in 1152 bytes} [Sat Jun 18 05:16:57 2022] POST /identity/v3/projects => generated 373 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:16:57.948643 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c65ca3a-0e22-46d6-9952-81120238d3c8 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:57.961582 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:DELETE=7,keystone:SELECT=91,keystone:INSERT=2 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:16:57.969517 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-e6c906a6-195f-42af-8ade-5c70fc9e8a84 None None] MFA Rules not processed for user `dfab4a1c0ff2430f9f234f790593b47c`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:16:57.970817 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-e6c906a6-195f-42af-8ade-5c70fc9e8a84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:16:58.145207 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 919/1804] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:16:57 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:16:58.150428 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c906a6-195f-42af-8ade-5c70fc9e8a84 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:58.213369 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c65ca3a-0e22-46d6-9952-81120238d3c8 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:58.229002 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e6de9e-5a40-4160-a977-6d98b5e54c06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:58.230523 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e6de9e-5a40-4160-a977-6d98b5e54c06 admin admin] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:58.231852 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e6de9e-5a40-4160-a977-6d98b5e54c06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:58.233111 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e6de9e-5a40-4160-a977-6d98b5e54c06 admin admin] PATH_INFO: `/v3/users` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:58.233874 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e6de9e-5a40-4160-a977-6d98b5e54c06 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:58.240680 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e6de9e-5a40-4160-a977-6d98b5e54c06 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:58.252863 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c906a6-195f-42af-8ade-5c70fc9e8a84 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:58.268046 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d67eced-4ad0-45a9-addc-b4e45dad04c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:58.269058 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d67eced-4ad0-45a9-addc-b4e45dad04c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:58.269583 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d67eced-4ad0-45a9-addc-b4e45dad04c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:58.270240 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d67eced-4ad0-45a9-addc-b4e45dad04c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:58.271282 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d67eced-4ad0-45a9-addc-b4e45dad04c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:58.273886 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d67eced-4ad0-45a9-addc-b4e45dad04c8 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:58.384363 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 886/1805] 10.222.0.23 () {60 vars in 1143 bytes} [Sat Jun 18 05:16:57 2022] POST /identity/v3/users => generated 287 bytes in 436 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:16:58.403539 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 920/1806] 10.222.0.23 () {60 vars in 1152 bytes} [Sat Jun 18 05:16:58 2022] POST /identity/v3/projects => generated 375 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:16:58.404606 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e6de9e-5a40-4160-a977-6d98b5e54c06 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:58.405369 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d67eced-4ad0-45a9-addc-b4e45dad04c8 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:58.449950 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e6de9e-5a40-4160-a977-6d98b5e54c06 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:58.543471 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:16:58.550524 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d67eced-4ad0-45a9-addc-b4e45dad04c8 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:58.558451 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f30a527-3ffb-4b43-874f-f870982e3757 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:58.561087 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f30a527-3ffb-4b43-874f-f870982e3757 admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:58.561087 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f30a527-3ffb-4b43-874f-f870982e3757 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:58.561087 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f30a527-3ffb-4b43-874f-f870982e3757 admin admin] PATH_INFO: `/v3/users` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:58.561087 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f30a527-3ffb-4b43-874f-f870982e3757 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:58.563452 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f30a527-3ffb-4b43-874f-f870982e3757 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:58.598852 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79896c25-e18f-40ca-a595-db25df3e2f2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:58.599548 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79896c25-e18f-40ca-a595-db25df3e2f2b admin admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:58.599841 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79896c25-e18f-40ca-a595-db25df3e2f2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:58.604942 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79896c25-e18f-40ca-a595-db25df3e2f2b admin admin] PATH_INFO: `/v3/roles` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:58.604942 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79896c25-e18f-40ca-a595-db25df3e2f2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:58.615561 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79896c25-e18f-40ca-a595-db25df3e2f2b admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:58.664111 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 887/1807] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:16:58 2022] GET /identity/v3/roles => generated 1420 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:16:58.671544 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79896c25-e18f-40ca-a595-db25df3e2f2b admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:58.683964 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 921/1808] 10.222.0.23 () {60 vars in 1143 bytes} [Sat Jun 18 05:16:58 2022] POST /identity/v3/users => generated 294 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:16:58.687579 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f30a527-3ffb-4b43-874f-f870982e3757 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:58.712428 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79896c25-e18f-40ca-a595-db25df3e2f2b admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:58.730901 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5f8814-9d4f-4d99-b94d-3c626309e528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:58.731666 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5f8814-9d4f-4d99-b94d-3c626309e528 admin admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:58.732015 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5f8814-9d4f-4d99-b94d-3c626309e528 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:58.732440 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5f8814-9d4f-4d99-b94d-3c626309e528 admin admin] PATH_INFO: `/v3/roles` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:58.733281 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5f8814-9d4f-4d99-b94d-3c626309e528 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:58.735741 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5f8814-9d4f-4d99-b94d-3c626309e528 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:58.737077 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f30a527-3ffb-4b43-874f-f870982e3757 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:58.760077 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebffc216-ec8e-42ab-8dda-cb0b65568c1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:58.761070 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebffc216-ec8e-42ab-8dda-cb0b65568c1c admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:58.761537 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebffc216-ec8e-42ab-8dda-cb0b65568c1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:58.761896 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebffc216-ec8e-42ab-8dda-cb0b65568c1c admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:58.765109 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 888/1809] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:16:58 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:16:58.768388 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebffc216-ec8e-42ab-8dda-cb0b65568c1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:58.770173 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5f8814-9d4f-4d99-b94d-3c626309e528 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:58.774929 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebffc216-ec8e-42ab-8dda-cb0b65568c1c admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:58.804560 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5f8814-9d4f-4d99-b94d-3c626309e528 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:58.813607 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 922/1810] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:16:58 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 18 05:16:58.814739 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebffc216-ec8e-42ab-8dda-cb0b65568c1c admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:58.820865 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014348e8-4722-426c-af28-449e178e89b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:58.822077 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014348e8-4722-426c-af28-449e178e89b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:58.822407 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014348e8-4722-426c-af28-449e178e89b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:58.822772 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014348e8-4722-426c-af28-449e178e89b0 admin admin] PATH_INFO: `/v3/projects/0e133bfc9c794233bcbc8635c8becfa2/users/2c0cf9e474aa44a59e90793a44061f35/roles/cbf5c4c26898413d93e73b46b41046a3` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:58.856267 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014348e8-4722-426c-af28-449e178e89b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e133bfc9c794233bcbc8635c8becfa2, user_id=2c0cf9e474aa44a59e90793a44061f35, role_id=cbf5c4c26898413d93e73b46b41046a3)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:58.859013 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014348e8-4722-426c-af28-449e178e89b0 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:58.863480 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebffc216-ec8e-42ab-8dda-cb0b65568c1c admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:58.885344 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1aaeee5-0f20-4bc0-9a49-2af479c1a55d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:58.886436 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aaeee5-0f20-4bc0-9a49-2af479c1a55d admin admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:58.886729 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aaeee5-0f20-4bc0-9a49-2af479c1a55d admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:58.887020 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aaeee5-0f20-4bc0-9a49-2af479c1a55d admin admin] PATH_INFO: `/v3/projects/40b318462cfd440e84949bccae41b40f/users/764d3fb762ed4bc48d144f319a68b7fc/roles/21c65f2c51bd4dd283d13507d2714467` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:58.926256 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1aaeee5-0f20-4bc0-9a49-2af479c1a55d admin admin] RBAC: Authorizing `identity:create_grant(project_id=40b318462cfd440e84949bccae41b40f, user_id=764d3fb762ed4bc48d144f319a68b7fc, role_id=21c65f2c51bd4dd283d13507d2714467)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:58.929489 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1aaeee5-0f20-4bc0-9a49-2af479c1a55d admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:58.947341 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 889/1811] 10.222.0.23 () {60 vars in 1482 bytes} [Sat Jun 18 05:16:58 2022] PUT /identity/v3/projects/0e133bfc9c794233bcbc8635c8becfa2/users/2c0cf9e474aa44a59e90793a44061f35/roles/cbf5c4c26898413d93e73b46b41046a3 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:16:58.960685 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8269d8c-028e-49cf-9a18-5b82ddaeb1dc None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:58.961116 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8269d8c-028e-49cf-9a18-5b82ddaeb1dc None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:58.961288 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8269d8c-028e-49cf-9a18-5b82ddaeb1dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:59.006882 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 923/1812] 10.222.0.23 () {60 vars in 1482 bytes} [Sat Jun 18 05:16:58 2022] PUT /identity/v3/projects/40b318462cfd440e84949bccae41b40f/users/764d3fb762ed4bc48d144f319a68b7fc/roles/21c65f2c51bd4dd283d13507d2714467 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:16:59.033874 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1aaeee5-0f20-4bc0-9a49-2af479c1a55d admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:59.060315 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-d8269d8c-028e-49cf-9a18-5b82ddaeb1dc None None] MFA Rules not processed for user `2c0cf9e474aa44a59e90793a44061f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:16:59.068392 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-d8269d8c-028e-49cf-9a18-5b82ddaeb1dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:16:59.213137 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:16:59.233988 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1aaeee5-0f20-4bc0-9a49-2af479c1a55d admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:59.250999 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd4ec26-f9d9-4f87-806d-af0d5dc6306d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:59.253061 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 890/1813] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:16:58 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:16:59.254128 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8269d8c-028e-49cf-9a18-5b82ddaeb1dc None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:59.255538 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-d8269d8c-028e-49cf-9a18-5b82ddaeb1dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:16:59.261438 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd4ec26-f9d9-4f87-806d-af0d5dc6306d admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:59.262947 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd4ec26-f9d9-4f87-806d-af0d5dc6306d admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:59.263255 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd4ec26-f9d9-4f87-806d-af0d5dc6306d admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:59.264704 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd4ec26-f9d9-4f87-806d-af0d5dc6306d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:59.270075 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd4ec26-f9d9-4f87-806d-af0d5dc6306d admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:59.279360 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8269d8c-028e-49cf-9a18-5b82ddaeb1dc None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:59.297365 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 924/1814] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:16:59 2022] GET /identity/v3/roles => generated 1420 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:16:59.306360 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd4ec26-f9d9-4f87-806d-af0d5dc6306d admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:59.307378 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b0ea90-d37f-4f40-bee5-48163d3186a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:59.308267 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b0ea90-d37f-4f40-bee5-48163d3186a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:59.308562 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b0ea90-d37f-4f40-bee5-48163d3186a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:59.308875 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b0ea90-d37f-4f40-bee5-48163d3186a5 admin admin] PATH_INFO: `/v3/projects/45c5494315cd497282ff00dd01b8ed9e` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:59.310605 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b0ea90-d37f-4f40-bee5-48163d3186a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=45c5494315cd497282ff00dd01b8ed9e)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:59.312712 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b0ea90-d37f-4f40-bee5-48163d3186a5 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:59.331555 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd4ec26-f9d9-4f87-806d-af0d5dc6306d admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:59.339449 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ee46e1-0141-4456-82ce-9c90d76c35e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:59.340544 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ee46e1-0141-4456-82ce-9c90d76c35e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:59.340832 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ee46e1-0141-4456-82ce-9c90d76c35e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:59.341423 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ee46e1-0141-4456-82ce-9c90d76c35e8 admin admin] PATH_INFO: `/v3/projects/40b318462cfd440e84949bccae41b40f/users/764d3fb762ed4bc48d144f319a68b7fc/roles/21c65f2c51bd4dd283d13507d2714467` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:59.344601 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ee46e1-0141-4456-82ce-9c90d76c35e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=40b318462cfd440e84949bccae41b40f, user_id=764d3fb762ed4bc48d144f319a68b7fc, role_id=21c65f2c51bd4dd283d13507d2714467)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:59.348650 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ee46e1-0141-4456-82ce-9c90d76c35e8 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:59.376359 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 925/1815] 10.222.0.23 () {60 vars in 1482 bytes} [Sat Jun 18 05:16:59 2022] PUT /identity/v3/projects/40b318462cfd440e84949bccae41b40f/users/764d3fb762ed4bc48d144f319a68b7fc/roles/21c65f2c51bd4dd283d13507d2714467 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:16:59.387698 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e74cf1-e948-450c-b7e4-cdb62c29b32d None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:59.387698 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e74cf1-e948-450c-b7e4-cdb62c29b32d None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:59.388221 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e74cf1-e948-450c-b7e4-cdb62c29b32d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:59.448630 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-f3e74cf1-e948-450c-b7e4-cdb62c29b32d None None] MFA Rules not processed for user `764d3fb762ed4bc48d144f319a68b7fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:16:59.453419 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-f3e74cf1-e948-450c-b7e4-cdb62c29b32d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:16:59.546535 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-24b0ea90-d37f-4f40-bee5-48163d3186a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '45c5494315cd497282ff00dd01b8ed9e', 'request_id': 'req-24b0ea90-d37f-4f40-bee5-48163d3186a5'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:16:59.565016 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-24b0ea90-d37f-4f40-bee5-48163d3186a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '45c5494315cd497282ff00dd01b8ed9e', 'request_id': 'req-24b0ea90-d37f-4f40-bee5-48163d3186a5'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:16:59.636869 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-24b0ea90-d37f-4f40-bee5-48163d3186a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '45c5494315cd497282ff00dd01b8ed9e', 'request_id': 'req-24b0ea90-d37f-4f40-bee5-48163d3186a5'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:16:59.649279 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 926/1816] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:16:59 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:16:59.650835 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e74cf1-e948-450c-b7e4-cdb62c29b32d None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:59.654512 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-f3e74cf1-e948-450c-b7e4-cdb62c29b32d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:16:59.662882 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-24b0ea90-d37f-4f40-bee5-48163d3186a5 admin admin] The token cache is being invalidate because project 45c5494315cd497282ff00dd01b8ed9e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80468) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 18 05:16:59.663376 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-24b0ea90-d37f-4f40-bee5-48163d3186a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:16:59.664847 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-24b0ea90-d37f-4f40-bee5-48163d3186a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:16:59.667376 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 891/1817] 10.222.0.23 () {58 vars in 1232 bytes} [Sat Jun 18 05:16:59 2022] DELETE /identity/v3/projects/45c5494315cd497282ff00dd01b8ed9e => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:16:59.676799 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b0ea90-d37f-4f40-bee5-48163d3186a5 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:16:59.679178 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-24b0ea90-d37f-4f40-bee5-48163d3186a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:16:59.735873 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:16:59.786229 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b0ea90-d37f-4f40-bee5-48163d3186a5 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:59.799020 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a50b96-81d3-414a-a7bf-2688aa5f64c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:59.800032 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a50b96-81d3-414a-a7bf-2688aa5f64c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:59.800320 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a50b96-81d3-414a-a7bf-2688aa5f64c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:59.800603 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a50b96-81d3-414a-a7bf-2688aa5f64c0 admin admin] PATH_INFO: `/v3/users/4554a3d40e7e458288f88832d83c56b2` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:59.802076 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a50b96-81d3-414a-a7bf-2688aa5f64c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4554a3d40e7e458288f88832d83c56b2)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:16:59.805422 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a50b96-81d3-414a-a7bf-2688aa5f64c0 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:16:59.814878 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e74cf1-e948-450c-b7e4-cdb62c29b32d None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:16:59.915245 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4756609e-054d-4c6c-9e35-ff22482b21cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9de0428abc8400aa338aba4f7c34c94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:16:59.915843 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4756609e-054d-4c6c-9e35-ff22482b21cd service cinder] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:16:59.916144 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4756609e-054d-4c6c-9e35-ff22482b21cd service cinder] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:16:59.916411 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4756609e-054d-4c6c-9e35-ff22482b21cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:16:59.918369 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-4756609e-054d-4c6c-9e35-ff22482b21cd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:17:00.040329 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4756609e-054d-4c6c-9e35-ff22482b21cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:00.042269 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4756609e-054d-4c6c-9e35-ff22482b21cd service cinder] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:00.053979 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 927/1818] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:16:59 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:17:00.055035 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4756609e-054d-4c6c-9e35-ff22482b21cd service cinder] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:00.056759 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-4756609e-054d-4c6c-9e35-ff22482b21cd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:17:00.078915 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4756609e-054d-4c6c-9e35-ff22482b21cd service cinder] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:00.085936 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ba19f7-46a9-4fda-9c51-5fb370072938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:00.087272 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ba19f7-46a9-4fda-9c51-5fb370072938 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:00.087650 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ba19f7-46a9-4fda-9c51-5fb370072938 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:00.087953 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ba19f7-46a9-4fda-9c51-5fb370072938 admin admin] PATH_INFO: `/v3/users/f90bd977ff8f4406bbcf63c4853bbf68` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:00.089502 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ba19f7-46a9-4fda-9c51-5fb370072938 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f90bd977ff8f4406bbcf63c4853bbf68)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:00.098514 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ba19f7-46a9-4fda-9c51-5fb370072938 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:00.098796 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-d3a50b96-81d3-414a-a7bf-2688aa5f64c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4554a3d40e7e458288f88832d83c56b2'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:00.098796 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-d3a50b96-81d3-414a-a7bf-2688aa5f64c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4554a3d40e7e458288f88832d83c56b2'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:00.201739 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-d3a50b96-81d3-414a-a7bf-2688aa5f64c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4554a3d40e7e458288f88832d83c56b2'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:00.217248 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-d3a50b96-81d3-414a-a7bf-2688aa5f64c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4554a3d40e7e458288f88832d83c56b2'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:00.219355 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:17:00.228012 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-d3a50b96-81d3-414a-a7bf-2688aa5f64c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4554a3d40e7e458288f88832d83c56b2'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:00.245856 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 892/1819] 10.222.0.23 () {58 vars in 1223 bytes} [Sat Jun 18 05:16:59 2022] DELETE /identity/v3/users/4554a3d40e7e458288f88832d83c56b2 => generated 0 bytes in 567 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:17:00.246901 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a50b96-81d3-414a-a7bf-2688aa5f64c0 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:00.248427 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-d3a50b96-81d3-414a-a7bf-2688aa5f64c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:17:00.318614 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-e8ba19f7-46a9-4fda-9c51-5fb370072938 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f90bd977ff8f4406bbcf63c4853bbf68'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:00.319535 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a50b96-81d3-414a-a7bf-2688aa5f64c0 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:00.324664 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590040bf-2e25-4ab1-b8c9-69bef72979b8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9de0428abc8400aa338aba4f7c34c94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:00.325539 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590040bf-2e25-4ab1-b8c9-69bef72979b8 service cinder] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:00.325941 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590040bf-2e25-4ab1-b8c9-69bef72979b8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:00.326347 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590040bf-2e25-4ab1-b8c9-69bef72979b8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:00.332790 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-590040bf-2e25-4ab1-b8c9-69bef72979b8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:17:00.348472 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-e8ba19f7-46a9-4fda-9c51-5fb370072938 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f90bd977ff8f4406bbcf63c4853bbf68'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:00.356025 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-e8ba19f7-46a9-4fda-9c51-5fb370072938 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f90bd977ff8f4406bbcf63c4853bbf68'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:00.425688 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-e8ba19f7-46a9-4fda-9c51-5fb370072938 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f90bd977ff8f4406bbcf63c4853bbf68'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:00.435193 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-e8ba19f7-46a9-4fda-9c51-5fb370072938 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f90bd977ff8f4406bbcf63c4853bbf68'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:00.438933 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:17:00.453991 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 928/1820] 10.222.0.23 () {58 vars in 1223 bytes} [Sat Jun 18 05:17:00 2022] DELETE /identity/v3/users/f90bd977ff8f4406bbcf63c4853bbf68 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:17:00.455001 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ba19f7-46a9-4fda-9c51-5fb370072938 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:00.458319 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-e8ba19f7-46a9-4fda-9c51-5fb370072938 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:17:00.463266 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590040bf-2e25-4ab1-b8c9-69bef72979b8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:00.465956 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590040bf-2e25-4ab1-b8c9-69bef72979b8 service cinder] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:00.497616 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 893/1821] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:17:00 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:17:00.499349 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43dc9ef-2a93-4bd3-ae43-b59396e301f3 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:00.499649 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43dc9ef-2a93-4bd3-ae43-b59396e301f3 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:00.499928 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43dc9ef-2a93-4bd3-ae43-b59396e301f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:00.544091 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-f43dc9ef-2a93-4bd3-ae43-b59396e301f3 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:17:00.545400 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-f43dc9ef-2a93-4bd3-ae43-b59396e301f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:17:00.626900 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ba19f7-46a9-4fda-9c51-5fb370072938 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:00.645303 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d2517f-6c6d-4638-aa3d-6ea007c6d6b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:00.648185 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d2517f-6c6d-4638-aa3d-6ea007c6d6b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:00.648185 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d2517f-6c6d-4638-aa3d-6ea007c6d6b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:00.648185 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d2517f-6c6d-4638-aa3d-6ea007c6d6b0 admin admin] PATH_INFO: `/v3/users/2c0cf9e474aa44a59e90793a44061f35` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:00.653401 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d2517f-6c6d-4638-aa3d-6ea007c6d6b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c0cf9e474aa44a59e90793a44061f35)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:00.655638 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d2517f-6c6d-4638-aa3d-6ea007c6d6b0 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:00.732768 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 894/1822] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:17:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:17:00.740589 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43dc9ef-2a93-4bd3-ae43-b59396e301f3 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:00.743432 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-f43dc9ef-2a93-4bd3-ae43-b59396e301f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:17:00.815665 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:17:00.844537 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43dc9ef-2a93-4bd3-ae43-b59396e301f3 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:00.853599 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21645369-607a-458c-8f0d-2789b1808001 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c4d7cde8f4ad9aa479bb09e325db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:00.854348 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21645369-607a-458c-8f0d-2789b1808001 service neutron] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:00.854720 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21645369-607a-458c-8f0d-2789b1808001 service neutron] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:00.857935 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21645369-607a-458c-8f0d-2789b1808001 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:00.883980 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21645369-607a-458c-8f0d-2789b1808001 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:00.883980 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21645369-607a-458c-8f0d-2789b1808001 service neutron] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:00.898367 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 895/1823] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:17:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:17:00.933981 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-07d2517f-6c6d-4638-aa3d-6ea007c6d6b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c0cf9e474aa44a59e90793a44061f35'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:00.952070 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-07d2517f-6c6d-4638-aa3d-6ea007c6d6b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c0cf9e474aa44a59e90793a44061f35'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:00.953105 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-07d2517f-6c6d-4638-aa3d-6ea007c6d6b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c0cf9e474aa44a59e90793a44061f35'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:01.025548 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-07d2517f-6c6d-4638-aa3d-6ea007c6d6b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c0cf9e474aa44a59e90793a44061f35'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:01.036844 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-07d2517f-6c6d-4638-aa3d-6ea007c6d6b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c0cf9e474aa44a59e90793a44061f35'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:01.052415 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 929/1824] 10.222.0.23 () {58 vars in 1223 bytes} [Sat Jun 18 05:17:00 2022] DELETE /identity/v3/users/2c0cf9e474aa44a59e90793a44061f35 => generated 0 bytes in 595 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:17:01.054154 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb59d886-ed45-4cf6-b7d7-3ac8040fd9d9 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:01.054395 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb59d886-ed45-4cf6-b7d7-3ac8040fd9d9 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:01.054704 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb59d886-ed45-4cf6-b7d7-3ac8040fd9d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:01.079906 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-cb59d886-ed45-4cf6-b7d7-3ac8040fd9d9 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:17:01.079906 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-cb59d886-ed45-4cf6-b7d7-3ac8040fd9d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:17:01.154348 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 896/1825] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:17:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 18 05:17:01.164296 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d2517f-6c6d-4638-aa3d-6ea007c6d6b0 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:01.169804 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-07d2517f-6c6d-4638-aa3d-6ea007c6d6b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:17:01.252430 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb59d886-ed45-4cf6-b7d7-3ac8040fd9d9 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:01.259076 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-cb59d886-ed45-4cf6-b7d7-3ac8040fd9d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:17:01.267816 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d2517f-6c6d-4638-aa3d-6ea007c6d6b0 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:01.273354 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07aaa30e-2872-4445-9b0c-5a504bb57b3c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c4d7cde8f4ad9aa479bb09e325db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:01.274556 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07aaa30e-2872-4445-9b0c-5a504bb57b3c service neutron] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:01.274912 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07aaa30e-2872-4445-9b0c-5a504bb57b3c service neutron] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:01.283443 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07aaa30e-2872-4445-9b0c-5a504bb57b3c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:01.295323 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07aaa30e-2872-4445-9b0c-5a504bb57b3c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:01.296848 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07aaa30e-2872-4445-9b0c-5a504bb57b3c service neutron] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:01.310209 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 930/1826] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:17:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 18 05:17:01.324089 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb59d886-ed45-4cf6-b7d7-3ac8040fd9d9 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:01.360996 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e50b10e-fdc8-4fd5-85f9-518837fd971b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:01.361963 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e50b10e-fdc8-4fd5-85f9-518837fd971b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:01.362252 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e50b10e-fdc8-4fd5-85f9-518837fd971b admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:01.365718 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e50b10e-fdc8-4fd5-85f9-518837fd971b admin admin] PATH_INFO: `/v3/projects/35413584bc0e486682d9f439fac5a887` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:01.367576 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e50b10e-fdc8-4fd5-85f9-518837fd971b admin admin] RBAC: Authorizing `identity:delete_project(project_id=35413584bc0e486682d9f439fac5a887)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:01.377124 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e50b10e-fdc8-4fd5-85f9-518837fd971b admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:01.556332 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-0e50b10e-fdc8-4fd5-85f9-518837fd971b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '35413584bc0e486682d9f439fac5a887', 'request_id': 'req-0e50b10e-fdc8-4fd5-85f9-518837fd971b'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:01.570280 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-0e50b10e-fdc8-4fd5-85f9-518837fd971b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '35413584bc0e486682d9f439fac5a887', 'request_id': 'req-0e50b10e-fdc8-4fd5-85f9-518837fd971b'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:01.571934 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07aaa30e-2872-4445-9b0c-5a504bb57b3c service neutron] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:01.573619 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-07aaa30e-2872-4445-9b0c-5a504bb57b3c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:17:01.596570 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07aaa30e-2872-4445-9b0c-5a504bb57b3c service neutron] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:01.624197 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:17:01.669895 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a0d53b-7d4d-41fe-8d3c-baff0439d32f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:01.670773 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a0d53b-7d4d-41fe-8d3c-baff0439d32f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:01.671206 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a0d53b-7d4d-41fe-8d3c-baff0439d32f admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:01.671488 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a0d53b-7d4d-41fe-8d3c-baff0439d32f admin admin] PATH_INFO: `/v3/projects/c8dc6016d2404e0e8fb4f303fa12427d` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:01.673030 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a0d53b-7d4d-41fe-8d3c-baff0439d32f admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8dc6016d2404e0e8fb4f303fa12427d)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:01.675019 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a0d53b-7d4d-41fe-8d3c-baff0439d32f admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:01.702378 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-0e50b10e-fdc8-4fd5-85f9-518837fd971b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '35413584bc0e486682d9f439fac5a887', 'request_id': 'req-0e50b10e-fdc8-4fd5-85f9-518837fd971b'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:01.720482 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-0e50b10e-fdc8-4fd5-85f9-518837fd971b admin admin] The token cache is being invalidate because project 35413584bc0e486682d9f439fac5a887 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80468) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 18 05:17:01.720782 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-0e50b10e-fdc8-4fd5-85f9-518837fd971b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:01.721521 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-0e50b10e-fdc8-4fd5-85f9-518837fd971b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:01.723201 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 897/1827] 10.222.0.23 () {58 vars in 1232 bytes} [Sat Jun 18 05:17:01 2022] DELETE /identity/v3/projects/35413584bc0e486682d9f439fac5a887 => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:17:01.728780 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd0e193-ae89-4db4-8699-e7ceff782270 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:01.729081 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd0e193-ae89-4db4-8699-e7ceff782270 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:01.729608 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd0e193-ae89-4db4-8699-e7ceff782270 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:01.749076 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-3dd0e193-ae89-4db4-8699-e7ceff782270 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:17:01.750318 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-3dd0e193-ae89-4db4-8699-e7ceff782270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:17:01.759452 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 898/1828] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:17:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 18 05:17:01.767818 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d80247-8f92-49d2-a284-3e25babe67fe None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:01.768290 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d80247-8f92-49d2-a284-3e25babe67fe None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:01.768585 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d80247-8f92-49d2-a284-3e25babe67fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:01.789209 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-45d80247-8f92-49d2-a284-3e25babe67fe None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:17:01.790688 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-45d80247-8f92-49d2-a284-3e25babe67fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:17:01.799591 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-22a0d53b-7d4d-41fe-8d3c-baff0439d32f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8dc6016d2404e0e8fb4f303fa12427d', 'request_id': 'req-22a0d53b-7d4d-41fe-8d3c-baff0439d32f'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:01.805754 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-22a0d53b-7d4d-41fe-8d3c-baff0439d32f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8dc6016d2404e0e8fb4f303fa12427d', 'request_id': 'req-22a0d53b-7d4d-41fe-8d3c-baff0439d32f'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:01.812194 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-22a0d53b-7d4d-41fe-8d3c-baff0439d32f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8dc6016d2404e0e8fb4f303fa12427d', 'request_id': 'req-22a0d53b-7d4d-41fe-8d3c-baff0439d32f'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:01.856429 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 899/1829] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:17:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:17:01.860877 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d80247-8f92-49d2-a284-3e25babe67fe None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:01.871819 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:17:01.876381 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d80247-8f92-49d2-a284-3e25babe67fe None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:01.883246 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca33064-92d5-430a-820f-ea63cba8888d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:01.883922 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca33064-92d5-430a-820f-ea63cba8888d admin admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:01.884207 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca33064-92d5-430a-820f-ea63cba8888d admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:01.884511 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca33064-92d5-430a-820f-ea63cba8888d admin admin] PATH_INFO: `/v3/domains` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:01.885249 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca33064-92d5-430a-820f-ea63cba8888d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 05:17:01.885608 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca33064-92d5-430a-820f-ea63cba8888d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:01.887309 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca33064-92d5-430a-820f-ea63cba8888d admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:01.900553 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 900/1830] 10.222.0.23 () {58 vars in 1152 bytes} [Sat Jun 18 05:17:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 18 05:17:01.901446 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca33064-92d5-430a-820f-ea63cba8888d admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:01.903447 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-bca33064-92d5-430a-820f-ea63cba8888d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:17:01.916445 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca33064-92d5-430a-820f-ea63cba8888d admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:01.921140 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18252630-edad-4759-980d-46640bd56c92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:01.924771 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18252630-edad-4759-980d-46640bd56c92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:01.924771 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18252630-edad-4759-980d-46640bd56c92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:01.924771 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18252630-edad-4759-980d-46640bd56c92 admin admin] PATH_INFO: `/v3/projects/0e133bfc9c794233bcbc8635c8becfa2` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:01.924771 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18252630-edad-4759-980d-46640bd56c92 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e133bfc9c794233bcbc8635c8becfa2)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:01.925694 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18252630-edad-4759-980d-46640bd56c92 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:01.926773 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-22a0d53b-7d4d-41fe-8d3c-baff0439d32f admin admin] The token cache is being invalidate because project c8dc6016d2404e0e8fb4f303fa12427d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80469) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 18 05:17:01.927063 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-22a0d53b-7d4d-41fe-8d3c-baff0439d32f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:01.927695 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-22a0d53b-7d4d-41fe-8d3c-baff0439d32f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:01.930134 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 931/1831] 10.222.0.23 () {58 vars in 1232 bytes} [Sat Jun 18 05:17:01 2022] DELETE /identity/v3/projects/c8dc6016d2404e0e8fb4f303fa12427d => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 18 05:17:01.931062 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a0d53b-7d4d-41fe-8d3c-baff0439d32f admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:01.932618 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-22a0d53b-7d4d-41fe-8d3c-baff0439d32f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:17:01.946028 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a0d53b-7d4d-41fe-8d3c-baff0439d32f admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:01.951879 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c70e2e-ea01-4c2f-93ff-61ed92ef7070 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:01.953080 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c70e2e-ea01-4c2f-93ff-61ed92ef7070 admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:01.953436 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c70e2e-ea01-4c2f-93ff-61ed92ef7070 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:01.953718 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c70e2e-ea01-4c2f-93ff-61ed92ef7070 admin admin] PATH_INFO: `/v3/projects` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:01.954898 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c70e2e-ea01-4c2f-93ff-61ed92ef7070 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:01.956806 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c70e2e-ea01-4c2f-93ff-61ed92ef7070 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:02.042498 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-18252630-edad-4759-980d-46640bd56c92 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e133bfc9c794233bcbc8635c8becfa2', 'request_id': 'req-18252630-edad-4759-980d-46640bd56c92'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:02.047772 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-18252630-edad-4759-980d-46640bd56c92 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e133bfc9c794233bcbc8635c8becfa2', 'request_id': 'req-18252630-edad-4759-980d-46640bd56c92'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:02.055575 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 932/1832] 10.222.0.23 () {60 vars in 1152 bytes} [Sat Jun 18 05:17:01 2022] POST /identity/v3/projects => generated 373 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:17:02.056539 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c70e2e-ea01-4c2f-93ff-61ed92ef7070 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:02.060090 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-65c70e2e-ea01-4c2f-93ff-61ed92ef7070 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:17:02.124193 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c70e2e-ea01-4c2f-93ff-61ed92ef7070 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:02.127643 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-18252630-edad-4759-980d-46640bd56c92 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e133bfc9c794233bcbc8635c8becfa2', 'request_id': 'req-18252630-edad-4759-980d-46640bd56c92'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:02.129324 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c14ffd9-d009-4b18-ad82-98eb1996ecbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:02.130851 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c14ffd9-d009-4b18-ad82-98eb1996ecbb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:02.131165 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c14ffd9-d009-4b18-ad82-98eb1996ecbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:02.131572 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c14ffd9-d009-4b18-ad82-98eb1996ecbb admin admin] PATH_INFO: `/v3/users/c204971ad01e48b7b071334717286d52` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:02.133812 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c14ffd9-d009-4b18-ad82-98eb1996ecbb admin admin] RBAC: Authorizing `identity:delete_user(user_id=c204971ad01e48b7b071334717286d52)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:02.136073 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c14ffd9-d009-4b18-ad82-98eb1996ecbb admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:02.153225 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-18252630-edad-4759-980d-46640bd56c92 admin admin] The token cache is being invalidate because project 0e133bfc9c794233bcbc8635c8becfa2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80468) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 18 05:17:02.153549 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-18252630-edad-4759-980d-46640bd56c92 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:02.154329 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-18252630-edad-4759-980d-46640bd56c92 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:02.159937 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 901/1833] 10.222.0.23 () {58 vars in 1232 bytes} [Sat Jun 18 05:17:01 2022] DELETE /identity/v3/projects/0e133bfc9c794233bcbc8635c8becfa2 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:17:02.161599 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18252630-edad-4759-980d-46640bd56c92 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:02.168120 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-18252630-edad-4759-980d-46640bd56c92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:17:02.242159 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18252630-edad-4759-980d-46640bd56c92 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:02.247151 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8a0451-7be0-4b1d-a086-edccfa9cc83d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:02.247611 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8a0451-7be0-4b1d-a086-edccfa9cc83d admin admin] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:02.247901 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8a0451-7be0-4b1d-a086-edccfa9cc83d admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:02.248182 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8a0451-7be0-4b1d-a086-edccfa9cc83d admin admin] PATH_INFO: `/v3/users` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:02.248876 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8a0451-7be0-4b1d-a086-edccfa9cc83d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:02.250895 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8a0451-7be0-4b1d-a086-edccfa9cc83d admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:02.300637 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-1c14ffd9-d009-4b18-ad82-98eb1996ecbb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c204971ad01e48b7b071334717286d52'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:02.313092 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-1c14ffd9-d009-4b18-ad82-98eb1996ecbb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c204971ad01e48b7b071334717286d52'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:02.313822 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-1c14ffd9-d009-4b18-ad82-98eb1996ecbb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c204971ad01e48b7b071334717286d52'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:02.318875 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 902/1834] 10.222.0.23 () {60 vars in 1143 bytes} [Sat Jun 18 05:17:02 2022] POST /identity/v3/users => generated 287 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:17:02.318875 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66319b5-1a11-4a0a-acd9-15941c993340 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:02.318875 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66319b5-1a11-4a0a-acd9-15941c993340 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:02.318875 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66319b5-1a11-4a0a-acd9-15941c993340 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:02.337457 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-e66319b5-1a11-4a0a-acd9-15941c993340 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:17:02.338744 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-e66319b5-1a11-4a0a-acd9-15941c993340 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:17:02.348033 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 903/1835] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:17:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:17:02.348991 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66319b5-1a11-4a0a-acd9-15941c993340 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:02.350868 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-e66319b5-1a11-4a0a-acd9-15941c993340 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:17:02.362331 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66319b5-1a11-4a0a-acd9-15941c993340 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:02.367559 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d2e9c3-4017-4f86-a775-791c194a9de3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:02.368200 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d2e9c3-4017-4f86-a775-791c194a9de3 admin admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:02.368483 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d2e9c3-4017-4f86-a775-791c194a9de3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:02.368759 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d2e9c3-4017-4f86-a775-791c194a9de3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:02.369571 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d2e9c3-4017-4f86-a775-791c194a9de3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:02.371155 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d2e9c3-4017-4f86-a775-791c194a9de3 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:02.381139 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 904/1836] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:17:02 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:17:02.382766 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8552b6-4f22-48a3-9489-496806510593 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:02.383082 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8552b6-4f22-48a3-9489-496806510593 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:02.383355 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8552b6-4f22-48a3-9489-496806510593 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:02.385277 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-1c14ffd9-d009-4b18-ad82-98eb1996ecbb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c204971ad01e48b7b071334717286d52'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:02.393997 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-1c14ffd9-d009-4b18-ad82-98eb1996ecbb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c204971ad01e48b7b071334717286d52'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:02.401128 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 933/1837] 10.222.0.23 () {58 vars in 1223 bytes} [Sat Jun 18 05:17:02 2022] DELETE /identity/v3/users/c204971ad01e48b7b071334717286d52 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:17:02.401489 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-6b8552b6-4f22-48a3-9489-496806510593 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:17:02.402203 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c14ffd9-d009-4b18-ad82-98eb1996ecbb admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:02.402745 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-6b8552b6-4f22-48a3-9489-496806510593 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:17:02.476247 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c14ffd9-d009-4b18-ad82-98eb1996ecbb admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:02.484442 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 905/1838] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:17:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:17:02.485736 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8552b6-4f22-48a3-9489-496806510593 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:02.498144 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8552b6-4f22-48a3-9489-496806510593 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:02.503363 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e1dec3-2b44-4a94-baf5-f23ee4fcb89a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:02.503363 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e1dec3-2b44-4a94-baf5-f23ee4fcb89a admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:02.503363 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e1dec3-2b44-4a94-baf5-f23ee4fcb89a admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:02.503363 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e1dec3-2b44-4a94-baf5-f23ee4fcb89a admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:02.503941 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e1dec3-2b44-4a94-baf5-f23ee4fcb89a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:02.504520 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a1657b-210e-47d8-a53c-912acf0a1bc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:02.505293 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a1657b-210e-47d8-a53c-912acf0a1bc5 admin admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:02.505697 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a1657b-210e-47d8-a53c-912acf0a1bc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:02.505899 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e1dec3-2b44-4a94-baf5-f23ee4fcb89a admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:02.506144 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a1657b-210e-47d8-a53c-912acf0a1bc5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:02.507047 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a1657b-210e-47d8-a53c-912acf0a1bc5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 05:17:02.507447 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a1657b-210e-47d8-a53c-912acf0a1bc5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:02.509976 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a1657b-210e-47d8-a53c-912acf0a1bc5 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:02.517555 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 934/1839] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:17:02 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:17:02.521545 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e1dec3-2b44-4a94-baf5-f23ee4fcb89a admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:02.533141 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e1dec3-2b44-4a94-baf5-f23ee4fcb89a admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:02.541423 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 906/1840] 10.222.0.23 () {58 vars in 1152 bytes} [Sat Jun 18 05:17:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 05:17:02.541611 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9959fd7d-d3c8-41be-a717-29dfefb85da0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:02.541901 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9959fd7d-d3c8-41be-a717-29dfefb85da0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:02.542275 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9959fd7d-d3c8-41be-a717-29dfefb85da0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:02.542645 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9959fd7d-d3c8-41be-a717-29dfefb85da0 admin admin] PATH_INFO: `/v3/projects/ceb4ebf6b13149448546688ff0599c6c/users/331d59a2875741838b6502c3cea39bd8/roles/cbf5c4c26898413d93e73b46b41046a3` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:02.544349 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a1657b-210e-47d8-a53c-912acf0a1bc5 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:02.555542 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a1657b-210e-47d8-a53c-912acf0a1bc5 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:02.560293 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a23a31b-577b-4699-b33d-95a9a1b3fe8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:02.560521 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9959fd7d-d3c8-41be-a717-29dfefb85da0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ceb4ebf6b13149448546688ff0599c6c, user_id=331d59a2875741838b6502c3cea39bd8, role_id=cbf5c4c26898413d93e73b46b41046a3)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:02.561244 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a23a31b-577b-4699-b33d-95a9a1b3fe8f admin admin] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:02.561571 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a23a31b-577b-4699-b33d-95a9a1b3fe8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:02.561847 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a23a31b-577b-4699-b33d-95a9a1b3fe8f admin admin] PATH_INFO: `/v3/projects` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:02.562535 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a23a31b-577b-4699-b33d-95a9a1b3fe8f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:02.563200 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9959fd7d-d3c8-41be-a717-29dfefb85da0 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:02.570615 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a23a31b-577b-4699-b33d-95a9a1b3fe8f admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:02.633372 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:17:02.640623 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 935/1841] 10.222.0.23 () {60 vars in 1482 bytes} [Sat Jun 18 05:17:02 2022] PUT /identity/v3/projects/ceb4ebf6b13149448546688ff0599c6c/users/331d59a2875741838b6502c3cea39bd8/roles/cbf5c4c26898413d93e73b46b41046a3 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 18 05:17:02.645722 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d0dbff-f8cf-4f57-8a44-749e7397cd79 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:02.646077 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d0dbff-f8cf-4f57-8a44-749e7397cd79 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:02.646445 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d0dbff-f8cf-4f57-8a44-749e7397cd79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:02.669278 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-f9d0dbff-f8cf-4f57-8a44-749e7397cd79 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:17:02.670895 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-f9d0dbff-f8cf-4f57-8a44-749e7397cd79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:17:02.688468 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:17:02.689101 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 907/1842] 10.222.0.23 () {60 vars in 1152 bytes} [Sat Jun 18 05:17:02 2022] POST /identity/v3/projects => generated 375 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 18 05:17:02.691407 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a23a31b-577b-4699-b33d-95a9a1b3fe8f admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:02.783423 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 936/1843] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:17:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 18 05:17:02.793751 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a23a31b-577b-4699-b33d-95a9a1b3fe8f admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:02.798694 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d0dbff-f8cf-4f57-8a44-749e7397cd79 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:02.799616 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1927ef-cbde-43a9-8d9e-4e17e8aed837 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:02.800300 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1927ef-cbde-43a9-8d9e-4e17e8aed837 admin admin] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:02.800406 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-f9d0dbff-f8cf-4f57-8a44-749e7397cd79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:17:02.800584 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1927ef-cbde-43a9-8d9e-4e17e8aed837 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:02.801212 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1927ef-cbde-43a9-8d9e-4e17e8aed837 admin admin] PATH_INFO: `/v3/users` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:02.802247 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1927ef-cbde-43a9-8d9e-4e17e8aed837 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:02.804036 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1927ef-cbde-43a9-8d9e-4e17e8aed837 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:02.866652 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d0dbff-f8cf-4f57-8a44-749e7397cd79 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:02.916073 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96162e4-aed3-4429-81fe-c92897550b6b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c4d7cde8f4ad9aa479bb09e325db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:02.916721 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96162e4-aed3-4429-81fe-c92897550b6b service neutron] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:02.917005 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96162e4-aed3-4429-81fe-c92897550b6b service neutron] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:02.917277 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96162e4-aed3-4429-81fe-c92897550b6b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:02.926490 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96162e4-aed3-4429-81fe-c92897550b6b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:02.927943 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96162e4-aed3-4429-81fe-c92897550b6b service neutron] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:02.936608 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 908/1844] 10.222.0.23 () {60 vars in 1143 bytes} [Sat Jun 18 05:17:02 2022] POST /identity/v3/users => generated 288 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:17:02.938325 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1927ef-cbde-43a9-8d9e-4e17e8aed837 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:02.939256 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 937/1845] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:17:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:17:02.940514 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-ab1927ef-cbde-43a9-8d9e-4e17e8aed837 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:17:02.940725 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96162e4-aed3-4429-81fe-c92897550b6b service neutron] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:02.952797 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96162e4-aed3-4429-81fe-c92897550b6b service neutron] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:02.960648 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166b4d4f-dac5-4b92-9a06-c84cbadef84a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:02.961348 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166b4d4f-dac5-4b92-9a06-c84cbadef84a admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:02.961631 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1927ef-cbde-43a9-8d9e-4e17e8aed837 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:02.961771 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166b4d4f-dac5-4b92-9a06-c84cbadef84a admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:02.961991 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166b4d4f-dac5-4b92-9a06-c84cbadef84a admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:02.962660 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166b4d4f-dac5-4b92-9a06-c84cbadef84a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:02.964298 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166b4d4f-dac5-4b92-9a06-c84cbadef84a admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:02.967554 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2d3db1-e34c-408f-9d6e-7b3f522dba31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:02.968426 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2d3db1-e34c-408f-9d6e-7b3f522dba31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:02.968797 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2d3db1-e34c-408f-9d6e-7b3f522dba31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:02.976637 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2d3db1-e34c-408f-9d6e-7b3f522dba31 admin admin] PATH_INFO: `/v3/projects/98533cb841214d4c9560c4e457b41b7e` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:02.980158 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 938/1846] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:17:02 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:17:02.980286 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2d3db1-e34c-408f-9d6e-7b3f522dba31 admin admin] RBAC: Authorizing `identity:delete_project(project_id=98533cb841214d4c9560c4e457b41b7e)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:02.980286 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2d3db1-e34c-408f-9d6e-7b3f522dba31 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:02.981660 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166b4d4f-dac5-4b92-9a06-c84cbadef84a admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:02.992894 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166b4d4f-dac5-4b92-9a06-c84cbadef84a admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:02.997762 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e5202a-b6f9-480c-af66-4022bc2cf26b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:02.998463 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e5202a-b6f9-480c-af66-4022bc2cf26b admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:02.998757 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e5202a-b6f9-480c-af66-4022bc2cf26b admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:02.999028 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e5202a-b6f9-480c-af66-4022bc2cf26b admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:02.999784 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e5202a-b6f9-480c-af66-4022bc2cf26b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:03.001757 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e5202a-b6f9-480c-af66-4022bc2cf26b admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:03.016445 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 939/1847] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:17:02 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:17:03.020590 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e5202a-b6f9-480c-af66-4022bc2cf26b admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:03.034094 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e5202a-b6f9-480c-af66-4022bc2cf26b admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:03.039835 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a73eb5c-fabf-424d-86a2-55d03689c2f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:03.040748 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a73eb5c-fabf-424d-86a2-55d03689c2f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:03.044660 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a73eb5c-fabf-424d-86a2-55d03689c2f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:03.044660 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a73eb5c-fabf-424d-86a2-55d03689c2f1 admin admin] PATH_INFO: `/v3/projects/469a82384ecf4afaaeb572287fc49fb8/users/f2c7241a70e44a87b556f471830b8f4c/roles/cbf5c4c26898413d93e73b46b41046a3` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:03.070930 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a73eb5c-fabf-424d-86a2-55d03689c2f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=469a82384ecf4afaaeb572287fc49fb8, user_id=f2c7241a70e44a87b556f471830b8f4c, role_id=cbf5c4c26898413d93e73b46b41046a3)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:03.074644 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a73eb5c-fabf-424d-86a2-55d03689c2f1 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:03.197310 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 940/1848] 10.222.0.23 () {60 vars in 1482 bytes} [Sat Jun 18 05:17:03 2022] PUT /identity/v3/projects/469a82384ecf4afaaeb572287fc49fb8/users/f2c7241a70e44a87b556f471830b8f4c/roles/cbf5c4c26898413d93e73b46b41046a3 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:17:03.208913 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0522c508-2b30-4f1d-aaa3-83e04bc6ab1d None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:03.209228 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0522c508-2b30-4f1d-aaa3-83e04bc6ab1d None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:03.209663 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0522c508-2b30-4f1d-aaa3-83e04bc6ab1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:03.223174 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-2b2d3db1-e34c-408f-9d6e-7b3f522dba31 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '98533cb841214d4c9560c4e457b41b7e', 'request_id': 'req-2b2d3db1-e34c-408f-9d6e-7b3f522dba31'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:03.230200 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-2b2d3db1-e34c-408f-9d6e-7b3f522dba31 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '98533cb841214d4c9560c4e457b41b7e', 'request_id': 'req-2b2d3db1-e34c-408f-9d6e-7b3f522dba31'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:03.248307 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:17:03.249525 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-0522c508-2b30-4f1d-aaa3-83e04bc6ab1d None None] MFA Rules not processed for user `f2c7241a70e44a87b556f471830b8f4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:17:03.251015 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-0522c508-2b30-4f1d-aaa3-83e04bc6ab1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:17:03.323004 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-2b2d3db1-e34c-408f-9d6e-7b3f522dba31 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '98533cb841214d4c9560c4e457b41b7e', 'request_id': 'req-2b2d3db1-e34c-408f-9d6e-7b3f522dba31'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:03.342366 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-2b2d3db1-e34c-408f-9d6e-7b3f522dba31 admin admin] The token cache is being invalidate because project 98533cb841214d4c9560c4e457b41b7e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80468) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 18 05:17:03.342665 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-2b2d3db1-e34c-408f-9d6e-7b3f522dba31 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:03.345642 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-2b2d3db1-e34c-408f-9d6e-7b3f522dba31 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:03.347649 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 909/1849] 10.222.0.23 () {58 vars in 1232 bytes} [Sat Jun 18 05:17:02 2022] DELETE /identity/v3/projects/98533cb841214d4c9560c4e457b41b7e => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:17:03.353512 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920efae9-ae74-40bf-ac0a-cc23c72dfba6 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:03.361270 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920efae9-ae74-40bf-ac0a-cc23c72dfba6 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:03.361270 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920efae9-ae74-40bf-ac0a-cc23c72dfba6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:03.385026 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-920efae9-ae74-40bf-ac0a-cc23c72dfba6 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:17:03.386278 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-920efae9-ae74-40bf-ac0a-cc23c72dfba6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:17:03.396353 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 941/1850] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:17:03 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:17:03.410850 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0522c508-2b30-4f1d-aaa3-83e04bc6ab1d None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:03.412322 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-0522c508-2b30-4f1d-aaa3-83e04bc6ab1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:17:03.455994 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:17:03.498194 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0522c508-2b30-4f1d-aaa3-83e04bc6ab1d None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:03.502815 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16279c62-aa62-4e1f-8622-20e71632e59a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9de0428abc8400aa338aba4f7c34c94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:03.503612 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16279c62-aa62-4e1f-8622-20e71632e59a service cinder] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:03.504820 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16279c62-aa62-4e1f-8622-20e71632e59a service cinder] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:03.505189 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16279c62-aa62-4e1f-8622-20e71632e59a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:03.522876 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16279c62-aa62-4e1f-8622-20e71632e59a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:03.525625 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16279c62-aa62-4e1f-8622-20e71632e59a service cinder] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:03.552089 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 942/1851] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:17:03 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:17:03.568453 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 910/1852] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:17:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 18 05:17:03.576663 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e235a1-f9e1-462a-9655-2f549e442cf7 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:03.577023 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e235a1-f9e1-462a-9655-2f549e442cf7 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:03.577302 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e235a1-f9e1-462a-9655-2f549e442cf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:03.623386 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-b5e235a1-f9e1-462a-9655-2f549e442cf7 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:17:03.631018 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-b5e235a1-f9e1-462a-9655-2f549e442cf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:17:03.638384 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 943/1853] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:17:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:17:03.639271 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920efae9-ae74-40bf-ac0a-cc23c72dfba6 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:03.651069 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920efae9-ae74-40bf-ac0a-cc23c72dfba6 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:03.656972 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8132a03-ff9e-496d-af8e-1fb41cb99134 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:03.658120 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8132a03-ff9e-496d-af8e-1fb41cb99134 admin admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:03.658458 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8132a03-ff9e-496d-af8e-1fb41cb99134 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:03.658793 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8132a03-ff9e-496d-af8e-1fb41cb99134 admin admin] PATH_INFO: `/v3/domains` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:03.659994 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8132a03-ff9e-496d-af8e-1fb41cb99134 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 05:17:03.660327 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8132a03-ff9e-496d-af8e-1fb41cb99134 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:03.662683 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8132a03-ff9e-496d-af8e-1fb41cb99134 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:03.681442 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 911/1854] 10.222.0.23 () {58 vars in 1152 bytes} [Sat Jun 18 05:17:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 18 05:17:03.685644 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e235a1-f9e1-462a-9655-2f549e442cf7 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:03.697043 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e235a1-f9e1-462a-9655-2f549e442cf7 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:03.703270 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a856a0-384b-41aa-83b7-729bde3d5f8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:03.703941 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a856a0-384b-41aa-83b7-729bde3d5f8c admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:03.707485 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a856a0-384b-41aa-83b7-729bde3d5f8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:03.707485 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a856a0-384b-41aa-83b7-729bde3d5f8c admin admin] PATH_INFO: `/v3/projects` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:03.707485 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a856a0-384b-41aa-83b7-729bde3d5f8c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:03.707715 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a856a0-384b-41aa-83b7-729bde3d5f8c admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:03.835618 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 944/1855] 10.222.0.23 () {60 vars in 1152 bytes} [Sat Jun 18 05:17:03 2022] POST /identity/v3/projects => generated 363 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:17:03.839616 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8132a03-ff9e-496d-af8e-1fb41cb99134 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:03.860641 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8132a03-ff9e-496d-af8e-1fb41cb99134 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:04.006149 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664c79c9-0474-42c8-a83e-6ea43f46b544 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:04.007550 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664c79c9-0474-42c8-a83e-6ea43f46b544 admin admin] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:04.010894 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664c79c9-0474-42c8-a83e-6ea43f46b544 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:04.010894 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664c79c9-0474-42c8-a83e-6ea43f46b544 admin admin] PATH_INFO: `/v3/users` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:04.010894 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664c79c9-0474-42c8-a83e-6ea43f46b544 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:04.026850 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664c79c9-0474-42c8-a83e-6ea43f46b544 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:04.153836 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 912/1856] 10.222.0.23 () {60 vars in 1143 bytes} [Sat Jun 18 05:17:03 2022] POST /identity/v3/users => generated 282 bytes in 314 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:17:04.163982 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a856a0-384b-41aa-83b7-729bde3d5f8c admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:04.176106 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a856a0-384b-41aa-83b7-729bde3d5f8c admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:04.191397 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7eec15-9ab5-479f-b36d-636f60873522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:04.192273 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7eec15-9ab5-479f-b36d-636f60873522 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:04.192590 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7eec15-9ab5-479f-b36d-636f60873522 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:04.193269 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7eec15-9ab5-479f-b36d-636f60873522 admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:04.194694 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7eec15-9ab5-479f-b36d-636f60873522 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:04.196356 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7eec15-9ab5-479f-b36d-636f60873522 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:04.215188 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 945/1857] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:17:04 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:17:04.219564 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664c79c9-0474-42c8-a83e-6ea43f46b544 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:04.240638 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664c79c9-0474-42c8-a83e-6ea43f46b544 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:04.247665 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1ac9df-742b-4bb8-b2c5-2b873b81f846 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:04.248476 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1ac9df-742b-4bb8-b2c5-2b873b81f846 admin admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:04.248769 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1ac9df-742b-4bb8-b2c5-2b873b81f846 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:04.249065 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1ac9df-742b-4bb8-b2c5-2b873b81f846 admin admin] PATH_INFO: `/v3/roles` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:04.249853 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1ac9df-742b-4bb8-b2c5-2b873b81f846 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:04.251630 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1ac9df-742b-4bb8-b2c5-2b873b81f846 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:04.268022 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 913/1858] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:17:04 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:17:04.268681 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7eec15-9ab5-479f-b36d-636f60873522 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:04.282003 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7eec15-9ab5-479f-b36d-636f60873522 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:04.295032 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea35eb2-d30e-4421-9800-dae9a4cfbfbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:04.296023 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea35eb2-d30e-4421-9800-dae9a4cfbfbd admin admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:04.296456 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea35eb2-d30e-4421-9800-dae9a4cfbfbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:04.296695 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea35eb2-d30e-4421-9800-dae9a4cfbfbd admin admin] PATH_INFO: `/v3/projects/bc024382b15c40e2bbb7cf58f1906fb9/users/c55c55e90f2b4a73bbdf3e9ffaf9a75f/roles/cbf5c4c26898413d93e73b46b41046a3` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:04.319546 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:17:04.325467 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea35eb2-d30e-4421-9800-dae9a4cfbfbd admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc024382b15c40e2bbb7cf58f1906fb9, user_id=c55c55e90f2b4a73bbdf3e9ffaf9a75f, role_id=cbf5c4c26898413d93e73b46b41046a3)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:04.328999 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea35eb2-d30e-4421-9800-dae9a4cfbfbd admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:04.471618 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 946/1859] 10.222.0.23 () {60 vars in 1482 bytes} [Sat Jun 18 05:17:04 2022] PUT /identity/v3/projects/bc024382b15c40e2bbb7cf58f1906fb9/users/c55c55e90f2b4a73bbdf3e9ffaf9a75f/roles/cbf5c4c26898413d93e73b46b41046a3 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:17:04.502814 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ef3633-5b68-404b-bc20-403bc7c879dd None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:04.502814 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ef3633-5b68-404b-bc20-403bc7c879dd None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:04.502814 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ef3633-5b68-404b-bc20-403bc7c879dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:04.555191 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-11ef3633-5b68-404b-bc20-403bc7c879dd None None] MFA Rules not processed for user `c55c55e90f2b4a73bbdf3e9ffaf9a75f`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:17:04.555191 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-11ef3633-5b68-404b-bc20-403bc7c879dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:17:04.619690 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 914/1860] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:17:04 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:17:04.624256 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea35eb2-d30e-4421-9800-dae9a4cfbfbd admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:04.699868 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea35eb2-d30e-4421-9800-dae9a4cfbfbd admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:04.705244 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b2a30e-0015-4e17-9adf-fce0217bfed2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:04.705946 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b2a30e-0015-4e17-9adf-fce0217bfed2 admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:04.706232 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b2a30e-0015-4e17-9adf-fce0217bfed2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:04.706509 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b2a30e-0015-4e17-9adf-fce0217bfed2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:04.707205 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b2a30e-0015-4e17-9adf-fce0217bfed2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:04.710295 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b2a30e-0015-4e17-9adf-fce0217bfed2 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:04.841941 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 947/1861] 10.222.0.23 () {60 vars in 1152 bytes} [Sat Jun 18 05:17:04 2022] POST /identity/v3/projects => generated 363 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:17:04.845553 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ef3633-5b68-404b-bc20-403bc7c879dd None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:04.861674 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ef3633-5b68-404b-bc20-403bc7c879dd None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:04.930006 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2472b6-bea2-4fff-83ba-0116cb64222c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:04.930721 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2472b6-bea2-4fff-83ba-0116cb64222c admin admin] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:04.931005 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2472b6-bea2-4fff-83ba-0116cb64222c admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:04.931282 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2472b6-bea2-4fff-83ba-0116cb64222c admin admin] PATH_INFO: `/v3/users` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:04.931971 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2472b6-bea2-4fff-83ba-0116cb64222c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:04.934072 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2472b6-bea2-4fff-83ba-0116cb64222c admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:04.962148 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:17:05.052256 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 915/1862] 10.222.0.23 () {60 vars in 1143 bytes} [Sat Jun 18 05:17:04 2022] POST /identity/v3/users => generated 288 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:17:05.054035 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b2a30e-0015-4e17-9adf-fce0217bfed2 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:05.075087 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b2a30e-0015-4e17-9adf-fce0217bfed2 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:05.080790 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efb3fb0-215d-4499-92e7-26fa356169e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:05.081505 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efb3fb0-215d-4499-92e7-26fa356169e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:05.081849 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efb3fb0-215d-4499-92e7-26fa356169e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:05.082136 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efb3fb0-215d-4499-92e7-26fa356169e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:05.082895 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efb3fb0-215d-4499-92e7-26fa356169e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:05.084560 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efb3fb0-215d-4499-92e7-26fa356169e2 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:05.099051 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 948/1863] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:17:05 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:17:05.103350 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2472b6-bea2-4fff-83ba-0116cb64222c admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:05.120219 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2472b6-bea2-4fff-83ba-0116cb64222c admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:05.120219 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48d239c-965e-491a-8928-ebd41050b163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:05.121214 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48d239c-965e-491a-8928-ebd41050b163 admin admin] REQUEST_METHOD: `PUT` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:05.121609 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48d239c-965e-491a-8928-ebd41050b163 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:05.122086 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48d239c-965e-491a-8928-ebd41050b163 admin admin] PATH_INFO: `/v3/projects/018458a4a37442dc975b845b4213e9b7/users/705f8a19f6254cff84510d9425aa3ce4/roles/21c65f2c51bd4dd283d13507d2714467` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:05.144811 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48d239c-965e-491a-8928-ebd41050b163 admin admin] RBAC: Authorizing `identity:create_grant(project_id=018458a4a37442dc975b845b4213e9b7, user_id=705f8a19f6254cff84510d9425aa3ce4, role_id=21c65f2c51bd4dd283d13507d2714467)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:05.149708 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48d239c-965e-491a-8928-ebd41050b163 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:05.162510 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efb3fb0-215d-4499-92e7-26fa356169e2 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:05.166145 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-4efb3fb0-215d-4499-92e7-26fa356169e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:17:05.179755 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efb3fb0-215d-4499-92e7-26fa356169e2 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:05.184687 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92603eb9-4578-4b20-abd1-de23c25a3f6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:05.185643 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92603eb9-4578-4b20-abd1-de23c25a3f6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:05.186807 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92603eb9-4578-4b20-abd1-de23c25a3f6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:05.186807 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92603eb9-4578-4b20-abd1-de23c25a3f6d admin admin] PATH_INFO: `/v3/users/dfab4a1c0ff2430f9f234f790593b47c` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:05.187460 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92603eb9-4578-4b20-abd1-de23c25a3f6d admin admin] RBAC: Authorizing `identity:delete_user(user_id=dfab4a1c0ff2430f9f234f790593b47c)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:05.189476 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92603eb9-4578-4b20-abd1-de23c25a3f6d admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:05.248698 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 916/1864] 10.222.0.23 () {60 vars in 1482 bytes} [Sat Jun 18 05:17:05 2022] PUT /identity/v3/projects/018458a4a37442dc975b845b4213e9b7/users/705f8a19f6254cff84510d9425aa3ce4/roles/21c65f2c51bd4dd283d13507d2714467 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:17:05.250563 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48d239c-965e-491a-8928-ebd41050b163 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:05.349241 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48d239c-965e-491a-8928-ebd41050b163 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:05.354974 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49027bac-534d-475a-864e-b64acde57f33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:05.355797 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49027bac-534d-475a-864e-b64acde57f33 admin admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:05.356027 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49027bac-534d-475a-864e-b64acde57f33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:05.356294 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49027bac-534d-475a-864e-b64acde57f33 admin admin] PATH_INFO: `/v3/roles` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:05.357032 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49027bac-534d-475a-864e-b64acde57f33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:05.358862 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49027bac-534d-475a-864e-b64acde57f33 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:05.373282 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 917/1865] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:17:05 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:17:05.377083 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49027bac-534d-475a-864e-b64acde57f33 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:05.396497 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49027bac-534d-475a-864e-b64acde57f33 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:05.405238 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e6d8b1-477a-4a62-bc6e-d68b8fff0ec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:05.412179 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e6d8b1-477a-4a62-bc6e-d68b8fff0ec4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:05.412179 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e6d8b1-477a-4a62-bc6e-d68b8fff0ec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:05.412179 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e6d8b1-477a-4a62-bc6e-d68b8fff0ec4 admin admin] PATH_INFO: `/v3/projects/018458a4a37442dc975b845b4213e9b7/users/705f8a19f6254cff84510d9425aa3ce4/roles/21c65f2c51bd4dd283d13507d2714467` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:05.413544 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e6d8b1-477a-4a62-bc6e-d68b8fff0ec4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=018458a4a37442dc975b845b4213e9b7, user_id=705f8a19f6254cff84510d9425aa3ce4, role_id=21c65f2c51bd4dd283d13507d2714467)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:05.416872 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e6d8b1-477a-4a62-bc6e-d68b8fff0ec4 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:05.445362 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 918/1866] 10.222.0.23 () {60 vars in 1482 bytes} [Sat Jun 18 05:17:05 2022] PUT /identity/v3/projects/018458a4a37442dc975b845b4213e9b7/users/705f8a19f6254cff84510d9425aa3ce4/roles/21c65f2c51bd4dd283d13507d2714467 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 18 05:17:05.446979 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caeabd3-c841-43cb-9fa2-bd0f4a447733 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:05.447276 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caeabd3-c841-43cb-9fa2-bd0f4a447733 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:05.447556 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caeabd3-c841-43cb-9fa2-bd0f4a447733 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:05.480350 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-92603eb9-4578-4b20-abd1-de23c25a3f6d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dfab4a1c0ff2430f9f234f790593b47c'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:05.520643 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-92603eb9-4578-4b20-abd1-de23c25a3f6d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dfab4a1c0ff2430f9f234f790593b47c'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:05.522180 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-92603eb9-4578-4b20-abd1-de23c25a3f6d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dfab4a1c0ff2430f9f234f790593b47c'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:05.531479 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-1caeabd3-c841-43cb-9fa2-bd0f4a447733 None None] MFA Rules not processed for user `705f8a19f6254cff84510d9425aa3ce4`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:17:05.536379 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-1caeabd3-c841-43cb-9fa2-bd0f4a447733 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:17:05.638605 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 919/1867] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:17:05 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:17:05.640575 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c0581c-a194-4cad-8046-4f37980da21d None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:05.641408 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c0581c-a194-4cad-8046-4f37980da21d None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:05.642094 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c0581c-a194-4cad-8046-4f37980da21d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:05.652903 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-92603eb9-4578-4b20-abd1-de23c25a3f6d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dfab4a1c0ff2430f9f234f790593b47c'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:05.665228 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-92603eb9-4578-4b20-abd1-de23c25a3f6d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dfab4a1c0ff2430f9f234f790593b47c'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:05.684738 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 949/1868] 10.222.0.23 () {58 vars in 1223 bytes} [Sat Jun 18 05:17:05 2022] DELETE /identity/v3/users/dfab4a1c0ff2430f9f234f790593b47c => generated 0 bytes in 523 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 18 05:17:05.691173 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9120e317-5ec4-4d5a-8e36-0fcd73ef826c None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:05.691648 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9120e317-5ec4-4d5a-8e36-0fcd73ef826c None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:05.692566 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9120e317-5ec4-4d5a-8e36-0fcd73ef826c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:05.736885 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-f5c0581c-a194-4cad-8046-4f37980da21d None None] MFA Rules not processed for user `331d59a2875741838b6502c3cea39bd8`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:17:05.739901 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-9120e317-5ec4-4d5a-8e36-0fcd73ef826c None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:17:05.741117 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-9120e317-5ec4-4d5a-8e36-0fcd73ef826c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:17:05.741902 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-f5c0581c-a194-4cad-8046-4f37980da21d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:17:05.868546 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 950/1869] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:17:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:17:05.871331 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9120e317-5ec4-4d5a-8e36-0fcd73ef826c None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:05.873485 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-9120e317-5ec4-4d5a-8e36-0fcd73ef826c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:17:05.923911 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 920/1870] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:17:05 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:17:05.924863 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c0581c-a194-4cad-8046-4f37980da21d None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:05.933477 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-f5c0581c-a194-4cad-8046-4f37980da21d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:17:05.987846 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9120e317-5ec4-4d5a-8e36-0fcd73ef826c None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:06.000432 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e61bef-fe33-4b76-98b5-0bd3b8d72674 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9de0428abc8400aa338aba4f7c34c94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:06.001151 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e61bef-fe33-4b76-98b5-0bd3b8d72674 service cinder] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:06.002033 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e61bef-fe33-4b76-98b5-0bd3b8d72674 service cinder] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:06.002319 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e61bef-fe33-4b76-98b5-0bd3b8d72674 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:06.076442 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c0581c-a194-4cad-8046-4f37980da21d None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:06.081635 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f50990-3c8e-4a7f-bb61-5a38e0ca52a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c4d7cde8f4ad9aa479bb09e325db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:06.082916 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f50990-3c8e-4a7f-bb61-5a38e0ca52a1 service neutron] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:06.084373 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f50990-3c8e-4a7f-bb61-5a38e0ca52a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:06.085948 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f50990-3c8e-4a7f-bb61-5a38e0ca52a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:06.104743 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f50990-3c8e-4a7f-bb61-5a38e0ca52a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:06.107092 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f50990-3c8e-4a7f-bb61-5a38e0ca52a1 service neutron] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:06.118274 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e61bef-fe33-4b76-98b5-0bd3b8d72674 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:06.120268 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 921/1871] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:17:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:17:06.121361 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f50990-3c8e-4a7f-bb61-5a38e0ca52a1 service neutron] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:06.122860 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e61bef-fe33-4b76-98b5-0bd3b8d72674 service cinder] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:06.123283 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-99f50990-3c8e-4a7f-bb61-5a38e0ca52a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:17:06.139500 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 951/1872] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:17:05 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:17:06.140907 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa0b0b9-000c-4a47-a63e-823dd1b28457 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:06.141424 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa0b0b9-000c-4a47-a63e-823dd1b28457 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:06.141702 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa0b0b9-000c-4a47-a63e-823dd1b28457 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:06.165881 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 952/1873] 10.222.0.23 () {54 vars in 921 bytes} [Sat Jun 18 05:17:06 2022] GET /identity => generated 270 bytes in 26 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 05:17:06.178098 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f50990-3c8e-4a7f-bb61-5a38e0ca52a1 service neutron] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:06.187399 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5f59a0-14c5-4eb0-933f-2b1fa8550c29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:06.188916 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5f59a0-14c5-4eb0-933f-2b1fa8550c29 admin admin] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:06.189548 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5f59a0-14c5-4eb0-933f-2b1fa8550c29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:06.190097 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5f59a0-14c5-4eb0-933f-2b1fa8550c29 admin admin] PATH_INFO: `/v3/projects` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:06.191369 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15172c53-a3ec-4322-828a-06d4642fb5fe None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:06.191837 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15172c53-a3ec-4322-828a-06d4642fb5fe None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:06.192092 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5f59a0-14c5-4eb0-933f-2b1fa8550c29 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:06.192228 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15172c53-a3ec-4322-828a-06d4642fb5fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:06.200135 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5f59a0-14c5-4eb0-933f-2b1fa8550c29 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:06.235400 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-15172c53-a3ec-4322-828a-06d4642fb5fe None None] MFA Rules not processed for user `15e7f45ee0744c1d9602187eac8e8b80`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:17:06.239569 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-15172c53-a3ec-4322-828a-06d4642fb5fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:17:06.339371 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 922/1874] 10.222.0.23 () {60 vars in 1152 bytes} [Sat Jun 18 05:17:06 2022] POST /identity/v3/projects => generated 373 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:17:06.343407 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5f59a0-14c5-4eb0-933f-2b1fa8550c29 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:06.372363 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 953/1875] 10.222.0.23 () {58 vars in 1018 bytes} [Sat Jun 18 05:17:06 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:17:06.390047 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15172c53-a3ec-4322-828a-06d4642fb5fe None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:06.391470 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-15172c53-a3ec-4322-828a-06d4642fb5fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:17:06.418662 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15172c53-a3ec-4322-828a-06d4642fb5fe None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:06.465894 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5f59a0-14c5-4eb0-933f-2b1fa8550c29 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:06.478242 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debacb8d-54a5-4351-9249-feba333c3889 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:06.479157 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debacb8d-54a5-4351-9249-feba333c3889 admin admin] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:06.479602 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debacb8d-54a5-4351-9249-feba333c3889 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:06.480026 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debacb8d-54a5-4351-9249-feba333c3889 admin admin] PATH_INFO: `/v3/users` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:06.480987 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debacb8d-54a5-4351-9249-feba333c3889 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:06.483109 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debacb8d-54a5-4351-9249-feba333c3889 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:06.501287 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ac0137-4c74-4096-9a32-baf26e643485 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15e7f45ee0744c1d9602187eac8e8b80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:06.502144 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ac0137-4c74-4096-9a32-baf26e643485 service nova] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:06.502432 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ac0137-4c74-4096-9a32-baf26e643485 service nova] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:06.502711 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ac0137-4c74-4096-9a32-baf26e643485 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:06.513890 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ac0137-4c74-4096-9a32-baf26e643485 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:06.515654 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ac0137-4c74-4096-9a32-baf26e643485 service nova] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:06.542974 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 954/1876] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:17:06 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:17:06.544002 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ac0137-4c74-4096-9a32-baf26e643485 service nova] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:06.575542 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 923/1877] 10.222.0.23 () {60 vars in 1143 bytes} [Sat Jun 18 05:17:06 2022] POST /identity/v3/users => generated 293 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:17:06.579521 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debacb8d-54a5-4351-9249-feba333c3889 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:06.597059 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debacb8d-54a5-4351-9249-feba333c3889 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:06.604342 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ac0137-4c74-4096-9a32-baf26e643485 service nova] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:06.609076 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfe3444-8a1c-4525-9974-021f1a766894 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9de0428abc8400aa338aba4f7c34c94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:06.609763 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfe3444-8a1c-4525-9974-021f1a766894 service cinder] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:06.610053 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfe3444-8a1c-4525-9974-021f1a766894 service cinder] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:06.610331 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfe3444-8a1c-4525-9974-021f1a766894 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:06.610866 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1bd465-a808-455e-a7f1-374da055993a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:06.611891 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1bd465-a808-455e-a7f1-374da055993a admin admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:06.612489 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1bd465-a808-455e-a7f1-374da055993a admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:06.612950 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1bd465-a808-455e-a7f1-374da055993a admin admin] PATH_INFO: `/v3/roles` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:06.613066 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-bcfe3444-8a1c-4525-9974-021f1a766894 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:17:06.614010 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1bd465-a808-455e-a7f1-374da055993a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:06.616506 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1bd465-a808-455e-a7f1-374da055993a admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:06.628089 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 924/1878] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:17:06 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 18 05:17:06.632517 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1bd465-a808-455e-a7f1-374da055993a admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:06.647649 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1bd465-a808-455e-a7f1-374da055993a admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:06.653275 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c81ee34-e131-4d36-82e6-9dea180ea3e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:06.657421 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c81ee34-e131-4d36-82e6-9dea180ea3e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:06.657421 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c81ee34-e131-4d36-82e6-9dea180ea3e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:06.657421 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c81ee34-e131-4d36-82e6-9dea180ea3e8 admin admin] PATH_INFO: `/v3/projects/9e7df6c98bc241398fb734d693186ca0/users/c7ba925fddb34ddc96af30fc06271861/roles/21c65f2c51bd4dd283d13507d2714467` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:06.671530 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c81ee34-e131-4d36-82e6-9dea180ea3e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e7df6c98bc241398fb734d693186ca0, user_id=c7ba925fddb34ddc96af30fc06271861, role_id=21c65f2c51bd4dd283d13507d2714467)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:06.674263 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c81ee34-e131-4d36-82e6-9dea180ea3e8 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:06.704785 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:17:06.708771 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfe3444-8a1c-4525-9974-021f1a766894 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:06.711292 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfe3444-8a1c-4525-9974-021f1a766894 service cinder] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:06.723063 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 955/1879] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:17:06 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:17:06.749151 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfe3444-8a1c-4525-9974-021f1a766894 service cinder] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:06.758250 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-bcfe3444-8a1c-4525-9974-021f1a766894 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:17:06.795458 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfe3444-8a1c-4525-9974-021f1a766894 service cinder] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:06.797701 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 925/1880] 10.222.0.23 () {60 vars in 1482 bytes} [Sat Jun 18 05:17:06 2022] PUT /identity/v3/projects/9e7df6c98bc241398fb734d693186ca0/users/c7ba925fddb34ddc96af30fc06271861/roles/21c65f2c51bd4dd283d13507d2714467 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 18 05:17:06.799019 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c81ee34-e131-4d36-82e6-9dea180ea3e8 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:06.888109 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3c0597-f196-4a77-92cf-08f965b85771 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:06.889107 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3c0597-f196-4a77-92cf-08f965b85771 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:06.889511 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3c0597-f196-4a77-92cf-08f965b85771 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:06.889862 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3c0597-f196-4a77-92cf-08f965b85771 admin admin] PATH_INFO: `/v3/projects/5dc0a53b17c54845883befbac80f3a7a` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:06.891431 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3c0597-f196-4a77-92cf-08f965b85771 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5dc0a53b17c54845883befbac80f3a7a)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:06.893531 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3c0597-f196-4a77-92cf-08f965b85771 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:06.918832 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c81ee34-e131-4d36-82e6-9dea180ea3e8 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:06.932153 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb86076-c797-4752-9132-94f7f13370f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:06.932852 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb86076-c797-4752-9132-94f7f13370f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:06.933140 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb86076-c797-4752-9132-94f7f13370f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:06.933785 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb86076-c797-4752-9132-94f7f13370f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:06.934679 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb86076-c797-4752-9132-94f7f13370f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:06.936483 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb86076-c797-4752-9132-94f7f13370f9 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:06.948709 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 926/1881] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:17:06 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 18 05:17:06.952918 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb86076-c797-4752-9132-94f7f13370f9 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:06.966652 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb86076-c797-4752-9132-94f7f13370f9 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:06.982057 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30f736e-57df-4c6b-bd37-b1f938c7b2a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:06.982057 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30f736e-57df-4c6b-bd37-b1f938c7b2a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:06.982442 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30f736e-57df-4c6b-bd37-b1f938c7b2a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:06.983024 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30f736e-57df-4c6b-bd37-b1f938c7b2a0 admin admin] PATH_INFO: `/v3/projects/9e7df6c98bc241398fb734d693186ca0/users/c7ba925fddb34ddc96af30fc06271861/roles/21c65f2c51bd4dd283d13507d2714467` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:06.987742 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30f736e-57df-4c6b-bd37-b1f938c7b2a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e7df6c98bc241398fb734d693186ca0, user_id=c7ba925fddb34ddc96af30fc06271861, role_id=21c65f2c51bd4dd283d13507d2714467)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:06.992099 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30f736e-57df-4c6b-bd37-b1f938c7b2a0 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:07.007813 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 927/1882] 10.222.0.23 () {60 vars in 1482 bytes} [Sat Jun 18 05:17:06 2022] PUT /identity/v3/projects/9e7df6c98bc241398fb734d693186ca0/users/c7ba925fddb34ddc96af30fc06271861/roles/21c65f2c51bd4dd283d13507d2714467 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 18 05:17:07.027492 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-3c3c0597-f196-4a77-92cf-08f965b85771 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5dc0a53b17c54845883befbac80f3a7a', 'request_id': 'req-3c3c0597-f196-4a77-92cf-08f965b85771'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:07.062146 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-3c3c0597-f196-4a77-92cf-08f965b85771 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5dc0a53b17c54845883befbac80f3a7a', 'request_id': 'req-3c3c0597-f196-4a77-92cf-08f965b85771'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:07.070939 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-3c3c0597-f196-4a77-92cf-08f965b85771 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5dc0a53b17c54845883befbac80f3a7a', 'request_id': 'req-3c3c0597-f196-4a77-92cf-08f965b85771'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:07.145946 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-3c3c0597-f196-4a77-92cf-08f965b85771 admin admin] The token cache is being invalidate because project 5dc0a53b17c54845883befbac80f3a7a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80469) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 18 05:17:07.146239 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-3c3c0597-f196-4a77-92cf-08f965b85771 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:07.146960 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-3c3c0597-f196-4a77-92cf-08f965b85771 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:07.150105 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 956/1883] 10.222.0.23 () {58 vars in 1232 bytes} [Sat Jun 18 05:17:06 2022] DELETE /identity/v3/projects/5dc0a53b17c54845883befbac80f3a7a => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 18 05:17:07.153770 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30f736e-57df-4c6b-bd37-b1f938c7b2a0 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:07.156217 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-f30f736e-57df-4c6b-bd37-b1f938c7b2a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:17:07.248171 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30f736e-57df-4c6b-bd37-b1f938c7b2a0 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:07.332167 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca39a84-eccf-474c-ae08-55134a00b342 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:07.333456 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca39a84-eccf-474c-ae08-55134a00b342 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:07.333757 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca39a84-eccf-474c-ae08-55134a00b342 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:07.334079 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca39a84-eccf-474c-ae08-55134a00b342 admin admin] PATH_INFO: `/v3/users/764d3fb762ed4bc48d144f319a68b7fc` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:07.335945 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca39a84-eccf-474c-ae08-55134a00b342 admin admin] RBAC: Authorizing `identity:delete_user(user_id=764d3fb762ed4bc48d144f319a68b7fc)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:07.338790 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca39a84-eccf-474c-ae08-55134a00b342 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:07.551967 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:17:07.626759 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-4ca39a84-eccf-474c-ae08-55134a00b342 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '764d3fb762ed4bc48d144f319a68b7fc'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:07.627422 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-4ca39a84-eccf-474c-ae08-55134a00b342 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '764d3fb762ed4bc48d144f319a68b7fc'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:07.705376 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-4ca39a84-eccf-474c-ae08-55134a00b342 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '764d3fb762ed4bc48d144f319a68b7fc'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:07.727047 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-4ca39a84-eccf-474c-ae08-55134a00b342 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '764d3fb762ed4bc48d144f319a68b7fc'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:07.736190 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-4ca39a84-eccf-474c-ae08-55134a00b342 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '764d3fb762ed4bc48d144f319a68b7fc'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:07.743798 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 928/1884] 10.222.0.23 () {58 vars in 1223 bytes} [Sat Jun 18 05:17:07 2022] DELETE /identity/v3/users/764d3fb762ed4bc48d144f319a68b7fc => generated 0 bytes in 590 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 18 05:17:08.206157 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3c0597-f196-4a77-92cf-08f965b85771 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:08.220381 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3c0597-f196-4a77-92cf-08f965b85771 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:08.294291 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fcf813-5381-4b1b-bc46-36d3a9f9c008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:08.295394 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fcf813-5381-4b1b-bc46-36d3a9f9c008 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:08.295922 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fcf813-5381-4b1b-bc46-36d3a9f9c008 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:08.296369 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fcf813-5381-4b1b-bc46-36d3a9f9c008 admin admin] PATH_INFO: `/v3/projects/40b318462cfd440e84949bccae41b40f` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:08.297796 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fcf813-5381-4b1b-bc46-36d3a9f9c008 admin admin] RBAC: Authorizing `identity:delete_project(project_id=40b318462cfd440e84949bccae41b40f)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:08.299833 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fcf813-5381-4b1b-bc46-36d3a9f9c008 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:08.406068 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-50fcf813-5381-4b1b-bc46-36d3a9f9c008 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40b318462cfd440e84949bccae41b40f', 'request_id': 'req-50fcf813-5381-4b1b-bc46-36d3a9f9c008'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:08.411962 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-50fcf813-5381-4b1b-bc46-36d3a9f9c008 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40b318462cfd440e84949bccae41b40f', 'request_id': 'req-50fcf813-5381-4b1b-bc46-36d3a9f9c008'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:08.419637 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-50fcf813-5381-4b1b-bc46-36d3a9f9c008 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40b318462cfd440e84949bccae41b40f', 'request_id': 'req-50fcf813-5381-4b1b-bc46-36d3a9f9c008'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:08.504948 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-50fcf813-5381-4b1b-bc46-36d3a9f9c008 admin admin] The token cache is being invalidate because project 40b318462cfd440e84949bccae41b40f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80469) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 18 05:17:08.505236 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-50fcf813-5381-4b1b-bc46-36d3a9f9c008 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:08.505892 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-50fcf813-5381-4b1b-bc46-36d3a9f9c008 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:08.507682 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 957/1885] 10.222.0.23 () {58 vars in 1232 bytes} [Sat Jun 18 05:17:08 2022] DELETE /identity/v3/projects/40b318462cfd440e84949bccae41b40f => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:17:08.512694 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15714502-106a-464d-987a-421c8be5d4ca None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:08.513167 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15714502-106a-464d-987a-421c8be5d4ca None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:08.513465 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15714502-106a-464d-987a-421c8be5d4ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:08.532086 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-15714502-106a-464d-987a-421c8be5d4ca None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:17:08.533566 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-15714502-106a-464d-987a-421c8be5d4ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:17:08.597114 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 929/1886] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:17:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:17:08.607242 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40cefb6-d095-4842-9f3a-8c92c83b3989 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:08.607557 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40cefb6-d095-4842-9f3a-8c92c83b3989 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:08.607799 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40cefb6-d095-4842-9f3a-8c92c83b3989 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:08.627507 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-b40cefb6-d095-4842-9f3a-8c92c83b3989 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:17:08.629014 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-b40cefb6-d095-4842-9f3a-8c92c83b3989 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:17:08.642425 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 958/1887] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:17:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 18 05:17:08.650853 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40cefb6-d095-4842-9f3a-8c92c83b3989 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:08.651483 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15714502-106a-464d-987a-421c8be5d4ca None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:08.652179 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-b40cefb6-d095-4842-9f3a-8c92c83b3989 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:17:08.664793 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15714502-106a-464d-987a-421c8be5d4ca None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:08.665319 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40cefb6-d095-4842-9f3a-8c92c83b3989 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:08.667902 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8287d0-7f19-42c7-a32c-bb04eba7a162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:08.668576 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8287d0-7f19-42c7-a32c-bb04eba7a162 admin admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:08.668863 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8287d0-7f19-42c7-a32c-bb04eba7a162 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:08.669142 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8287d0-7f19-42c7-a32c-bb04eba7a162 admin admin] PATH_INFO: `/v3/domains` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:08.670133 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8287d0-7f19-42c7-a32c-bb04eba7a162 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 05:17:08.670633 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8287d0-7f19-42c7-a32c-bb04eba7a162 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:08.670891 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7af9b29-4cf2-4e31-be0b-d0d6d256ac3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:08.671889 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7af9b29-4cf2-4e31-be0b-d0d6d256ac3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:08.672239 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7af9b29-4cf2-4e31-be0b-d0d6d256ac3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:08.672346 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8287d0-7f19-42c7-a32c-bb04eba7a162 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:08.672550 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7af9b29-4cf2-4e31-be0b-d0d6d256ac3a admin admin] PATH_INFO: `/v3/users/f2c7241a70e44a87b556f471830b8f4c` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:08.673949 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7af9b29-4cf2-4e31-be0b-d0d6d256ac3a admin admin] RBAC: Authorizing `identity:delete_user(user_id=f2c7241a70e44a87b556f471830b8f4c)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:08.687908 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 930/1888] 10.222.0.23 () {58 vars in 1152 bytes} [Sat Jun 18 05:17:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 05:17:08.688764 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7af9b29-4cf2-4e31-be0b-d0d6d256ac3a admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:08.698621 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8287d0-7f19-42c7-a32c-bb04eba7a162 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:08.721109 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8287d0-7f19-42c7-a32c-bb04eba7a162 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:08.732903 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac939ce-4856-4b23-8ea9-b2232f7db8e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:08.733753 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac939ce-4856-4b23-8ea9-b2232f7db8e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:08.734024 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac939ce-4856-4b23-8ea9-b2232f7db8e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:08.734306 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac939ce-4856-4b23-8ea9-b2232f7db8e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:08.735116 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac939ce-4856-4b23-8ea9-b2232f7db8e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:08.737251 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac939ce-4856-4b23-8ea9-b2232f7db8e8 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:08.814689 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 931/1889] 10.222.0.23 () {60 vars in 1152 bytes} [Sat Jun 18 05:17:08 2022] POST /identity/v3/projects => generated 383 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:17:08.816095 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213f3005-2a8d-4ce2-ae35-aebf46d5711a None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:08.816387 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213f3005-2a8d-4ce2-ae35-aebf46d5711a None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:08.816665 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213f3005-2a8d-4ce2-ae35-aebf46d5711a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:08.862576 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-213f3005-2a8d-4ce2-ae35-aebf46d5711a None None] MFA Rules not processed for user `c7ba925fddb34ddc96af30fc06271861`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:17:08.863988 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-213f3005-2a8d-4ce2-ae35-aebf46d5711a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:17:08.943161 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-f7af9b29-4cf2-4e31-be0b-d0d6d256ac3a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f2c7241a70e44a87b556f471830b8f4c'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:08.956752 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-f7af9b29-4cf2-4e31-be0b-d0d6d256ac3a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f2c7241a70e44a87b556f471830b8f4c'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:08.957931 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-f7af9b29-4cf2-4e31-be0b-d0d6d256ac3a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f2c7241a70e44a87b556f471830b8f4c'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:08.963778 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 932/1890] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:17:08 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:17:08.963778 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213f3005-2a8d-4ce2-ae35-aebf46d5711a None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:09.028201 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213f3005-2a8d-4ce2-ae35-aebf46d5711a None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:09.032987 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b4cbc0-ce80-419c-88b3-60aaa7b323b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:09.039463 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b4cbc0-ce80-419c-88b3-60aaa7b323b4 admin admin] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:09.039463 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b4cbc0-ce80-419c-88b3-60aaa7b323b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:09.039463 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b4cbc0-ce80-419c-88b3-60aaa7b323b4 admin admin] PATH_INFO: `/v3/users` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:09.039463 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b4cbc0-ce80-419c-88b3-60aaa7b323b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:09.039463 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b4cbc0-ce80-419c-88b3-60aaa7b323b4 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:09.059786 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-f7af9b29-4cf2-4e31-be0b-d0d6d256ac3a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f2c7241a70e44a87b556f471830b8f4c'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:09.071889 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-f7af9b29-4cf2-4e31-be0b-d0d6d256ac3a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f2c7241a70e44a87b556f471830b8f4c'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:09.082249 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 959/1891] 10.222.0.23 () {58 vars in 1223 bytes} [Sat Jun 18 05:17:08 2022] DELETE /identity/v3/users/f2c7241a70e44a87b556f471830b8f4c => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 18 05:17:09.083207 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7af9b29-4cf2-4e31-be0b-d0d6d256ac3a admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:09.084807 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-f7af9b29-4cf2-4e31-be0b-d0d6d256ac3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:17:09.135680 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 933/1892] 10.222.0.23 () {60 vars in 1143 bytes} [Sat Jun 18 05:17:08 2022] POST /identity/v3/users => generated 292 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:17:09.137093 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b258b5e-d402-4d4b-b0fd-5905731d4403 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:09.137411 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b258b5e-d402-4d4b-b0fd-5905731d4403 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:09.138020 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b258b5e-d402-4d4b-b0fd-5905731d4403 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:09.174993 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-1b258b5e-d402-4d4b-b0fd-5905731d4403 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:17:09.176196 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-1b258b5e-d402-4d4b-b0fd-5905731d4403 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:17:09.192753 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7af9b29-4cf2-4e31-be0b-d0d6d256ac3a admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:09.204613 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf3cde2-b0ed-4aab-bf94-ce91c25261d1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9de0428abc8400aa338aba4f7c34c94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:09.206103 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf3cde2-b0ed-4aab-bf94-ce91c25261d1 service cinder] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:09.209723 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf3cde2-b0ed-4aab-bf94-ce91c25261d1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:09.209723 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf3cde2-b0ed-4aab-bf94-ce91c25261d1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:09.211096 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-fdf3cde2-b0ed-4aab-bf94-ce91c25261d1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:17:09.246151 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 934/1893] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:17:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:17:09.246151 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b258b5e-d402-4d4b-b0fd-5905731d4403 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:09.246151 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-1b258b5e-d402-4d4b-b0fd-5905731d4403 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:17:09.259789 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b258b5e-d402-4d4b-b0fd-5905731d4403 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:09.265508 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc99ab8-8c29-4d40-ae44-cbc66cfa2e9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:09.266188 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc99ab8-8c29-4d40-ae44-cbc66cfa2e9d admin admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:09.266470 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc99ab8-8c29-4d40-ae44-cbc66cfa2e9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:09.266741 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc99ab8-8c29-4d40-ae44-cbc66cfa2e9d admin admin] PATH_INFO: `/v3/roles` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:09.267533 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc99ab8-8c29-4d40-ae44-cbc66cfa2e9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:09.269188 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc99ab8-8c29-4d40-ae44-cbc66cfa2e9d admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:09.281144 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 935/1894] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:17:09 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:17:09.284573 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc99ab8-8c29-4d40-ae44-cbc66cfa2e9d admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:09.285735 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-fcc99ab8-8c29-4d40-ae44-cbc66cfa2e9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:17:09.297915 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf3cde2-b0ed-4aab-bf94-ce91c25261d1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:09.315129 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf3cde2-b0ed-4aab-bf94-ce91c25261d1 service cinder] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:09.353173 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 960/1895] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:17:09 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:17:09.354399 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf3cde2-b0ed-4aab-bf94-ce91c25261d1 service cinder] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:09.357612 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc99ab8-8c29-4d40-ae44-cbc66cfa2e9d admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:09.365684 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e02584e-4284-4074-a698-6e87c6dfaa27 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c4d7cde8f4ad9aa479bb09e325db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:09.365684 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e02584e-4284-4074-a698-6e87c6dfaa27 service neutron] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:09.365684 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e02584e-4284-4074-a698-6e87c6dfaa27 service neutron] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:09.365684 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e02584e-4284-4074-a698-6e87c6dfaa27 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:09.378739 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e02584e-4284-4074-a698-6e87c6dfaa27 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:09.380090 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e02584e-4284-4074-a698-6e87c6dfaa27 service neutron] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:09.393677 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 936/1896] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:17:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:17:09.394796 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e02584e-4284-4074-a698-6e87c6dfaa27 service neutron] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:09.397898 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-0e02584e-4284-4074-a698-6e87c6dfaa27 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:17:09.402468 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf3cde2-b0ed-4aab-bf94-ce91c25261d1 service cinder] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:09.413798 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e02584e-4284-4074-a698-6e87c6dfaa27 service neutron] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:09.414574 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ad9a08-5113-40ab-897e-32b771a5fe00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:09.415245 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ad9a08-5113-40ab-897e-32b771a5fe00 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:09.415533 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ad9a08-5113-40ab-897e-32b771a5fe00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:09.415814 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ad9a08-5113-40ab-897e-32b771a5fe00 admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:09.417985 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ad9a08-5113-40ab-897e-32b771a5fe00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:09.420339 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51f1fc3-b77e-49a2-aacd-73559918cb7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:09.421329 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51f1fc3-b77e-49a2-aacd-73559918cb7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:09.421848 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51f1fc3-b77e-49a2-aacd-73559918cb7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:09.422126 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51f1fc3-b77e-49a2-aacd-73559918cb7d admin admin] PATH_INFO: `/v3/users/aa6ff0b2e3d343c0b2921d3cdb269066` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:09.423638 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51f1fc3-b77e-49a2-aacd-73559918cb7d admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa6ff0b2e3d343c0b2921d3cdb269066)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:09.424416 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ad9a08-5113-40ab-897e-32b771a5fe00 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:09.429298 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51f1fc3-b77e-49a2-aacd-73559918cb7d admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:09.450199 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 961/1897] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:17:09 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:17:09.451797 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ad9a08-5113-40ab-897e-32b771a5fe00 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:09.475723 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ad9a08-5113-40ab-897e-32b771a5fe00 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:09.489095 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e95676f-01e9-4bdf-82dc-a44e6ac52e4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:09.490027 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e95676f-01e9-4bdf-82dc-a44e6ac52e4b admin admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:09.490310 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e95676f-01e9-4bdf-82dc-a44e6ac52e4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:09.491292 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e95676f-01e9-4bdf-82dc-a44e6ac52e4b admin admin] PATH_INFO: `/v3/projects/b471c5c09317491eb8a3ac7f954ee544/users/fcf611fb28de427a91d438c9af2ed4c2/roles/cbf5c4c26898413d93e73b46b41046a3` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:09.542706 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e95676f-01e9-4bdf-82dc-a44e6ac52e4b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b471c5c09317491eb8a3ac7f954ee544, user_id=fcf611fb28de427a91d438c9af2ed4c2, role_id=cbf5c4c26898413d93e73b46b41046a3)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:09.551518 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e95676f-01e9-4bdf-82dc-a44e6ac52e4b admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:09.608508 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-f51f1fc3-b77e-49a2-aacd-73559918cb7d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa6ff0b2e3d343c0b2921d3cdb269066'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:09.608508 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-f51f1fc3-b77e-49a2-aacd-73559918cb7d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa6ff0b2e3d343c0b2921d3cdb269066'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:09.725435 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 962/1898] 10.222.0.23 () {60 vars in 1482 bytes} [Sat Jun 18 05:17:09 2022] PUT /identity/v3/projects/b471c5c09317491eb8a3ac7f954ee544/users/fcf611fb28de427a91d438c9af2ed4c2/roles/cbf5c4c26898413d93e73b46b41046a3 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:17:09.735968 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1d416d-9e0d-43f7-83bb-365fc8517bf2 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:09.750655 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1d416d-9e0d-43f7-83bb-365fc8517bf2 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:09.750655 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1d416d-9e0d-43f7-83bb-365fc8517bf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:09.767925 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-ea1d416d-9e0d-43f7-83bb-365fc8517bf2 None None] MFA Rules not processed for user `fcf611fb28de427a91d438c9af2ed4c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:17:09.769210 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-ea1d416d-9e0d-43f7-83bb-365fc8517bf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:17:09.846772 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 963/1899] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:17:09 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 18 05:17:09.848851 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9217306-2337-4f95-94e5-22060223ec86 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:09.849152 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9217306-2337-4f95-94e5-22060223ec86 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:09.849470 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9217306-2337-4f95-94e5-22060223ec86 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:09.852112 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 964/1900] 10.222.0.23 () {54 vars in 921 bytes} [Sat Jun 18 05:17:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 05:17:09.858796 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6c602f-40f9-4b71-acc9-06e84693c132 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:09.859086 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6c602f-40f9-4b71-acc9-06e84693c132 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:09.859369 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6c602f-40f9-4b71-acc9-06e84693c132 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:09.879710 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-4c6c602f-40f9-4b71-acc9-06e84693c132 None None] MFA Rules not processed for user `15e7f45ee0744c1d9602187eac8e8b80`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:17:09.880923 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-4c6c602f-40f9-4b71-acc9-06e84693c132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:17:09.952564 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 965/1901] 10.222.0.23 () {58 vars in 1018 bytes} [Sat Jun 18 05:17:09 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:17:09.962459 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6c602f-40f9-4b71-acc9-06e84693c132 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:09.963975 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-4c6c602f-40f9-4b71-acc9-06e84693c132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:17:10.035405 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6c602f-40f9-4b71-acc9-06e84693c132 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:10.040277 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e6d549-2fde-48ec-93f9-3e1883303d3f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9de0428abc8400aa338aba4f7c34c94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:10.040930 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e6d549-2fde-48ec-93f9-3e1883303d3f service cinder] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:10.041216 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e6d549-2fde-48ec-93f9-3e1883303d3f service cinder] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:10.041524 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e6d549-2fde-48ec-93f9-3e1883303d3f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:10.058504 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e6d549-2fde-48ec-93f9-3e1883303d3f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:10.060343 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e6d549-2fde-48ec-93f9-3e1883303d3f service cinder] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:10.073031 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 966/1902] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:17:09 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:17:10.115535 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e6d549-2fde-48ec-93f9-3e1883303d3f service cinder] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:10.115535 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-09e6d549-2fde-48ec-93f9-3e1883303d3f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:17:10.130111 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e6d549-2fde-48ec-93f9-3e1883303d3f service cinder] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:10.146611 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2dedda-070a-4f91-99fe-d382a8eb1e39 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15e7f45ee0744c1d9602187eac8e8b80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:10.149373 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2dedda-070a-4f91-99fe-d382a8eb1e39 service nova] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:10.150759 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2dedda-070a-4f91-99fe-d382a8eb1e39 service nova] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:10.151326 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2dedda-070a-4f91-99fe-d382a8eb1e39 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:10.172705 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2dedda-070a-4f91-99fe-d382a8eb1e39 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:10.175010 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2dedda-070a-4f91-99fe-d382a8eb1e39 service nova] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:10.216522 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 967/1903] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:17:10 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 18 05:17:10.338956 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-f51f1fc3-b77e-49a2-aacd-73559918cb7d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa6ff0b2e3d343c0b2921d3cdb269066'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:10.353835 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-f51f1fc3-b77e-49a2-aacd-73559918cb7d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa6ff0b2e3d343c0b2921d3cdb269066'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:10.365811 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-f51f1fc3-b77e-49a2-aacd-73559918cb7d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa6ff0b2e3d343c0b2921d3cdb269066'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:10.378623 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 937/1904] 10.222.0.23 () {58 vars in 1223 bytes} [Sat Jun 18 05:17:09 2022] DELETE /identity/v3/users/aa6ff0b2e3d343c0b2921d3cdb269066 => generated 0 bytes in 984 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:17:10.383747 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c503b1-a690-42da-8b04-620d3440de62 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:10.384135 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c503b1-a690-42da-8b04-620d3440de62 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:10.384734 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c503b1-a690-42da-8b04-620d3440de62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:10.411266 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-d0c503b1-a690-42da-8b04-620d3440de62 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:17:10.412580 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-d0c503b1-a690-42da-8b04-620d3440de62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:17:10.490085 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 968/1905] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:17:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 18 05:17:10.523057 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51f1fc3-b77e-49a2-aacd-73559918cb7d admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:10.525490 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-f51f1fc3-b77e-49a2-aacd-73559918cb7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:17:10.661020 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51f1fc3-b77e-49a2-aacd-73559918cb7d admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:10.671222 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f65d1c-1754-40ba-b785-d944a43af66f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c4d7cde8f4ad9aa479bb09e325db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:10.671941 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f65d1c-1754-40ba-b785-d944a43af66f service neutron] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:10.672223 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f65d1c-1754-40ba-b785-d944a43af66f service neutron] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:10.672503 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f65d1c-1754-40ba-b785-d944a43af66f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:10.697311 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f65d1c-1754-40ba-b785-d944a43af66f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:10.699411 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f65d1c-1754-40ba-b785-d944a43af66f service neutron] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:10.724165 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 938/1906] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:17:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:17:10.820007 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:17:10.956166 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c503b1-a690-42da-8b04-620d3440de62 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:10.959248 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-d0c503b1-a690-42da-8b04-620d3440de62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:17:10.979867 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c503b1-a690-42da-8b04-620d3440de62 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:10.986283 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8123ee4c-ebeb-4965-a560-fdff7a60ac85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:10.987631 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8123ee4c-ebeb-4965-a560-fdff7a60ac85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:10.987631 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8123ee4c-ebeb-4965-a560-fdff7a60ac85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:10.987849 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8123ee4c-ebeb-4965-a560-fdff7a60ac85 admin admin] PATH_INFO: `/v3/projects/469a82384ecf4afaaeb572287fc49fb8` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:10.989249 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8123ee4c-ebeb-4965-a560-fdff7a60ac85 admin admin] RBAC: Authorizing `identity:delete_project(project_id=469a82384ecf4afaaeb572287fc49fb8)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:10.991339 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8123ee4c-ebeb-4965-a560-fdff7a60ac85 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:11.141011 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-8123ee4c-ebeb-4965-a560-fdff7a60ac85 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '469a82384ecf4afaaeb572287fc49fb8', 'request_id': 'req-8123ee4c-ebeb-4965-a560-fdff7a60ac85'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:11.159525 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-8123ee4c-ebeb-4965-a560-fdff7a60ac85 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '469a82384ecf4afaaeb572287fc49fb8', 'request_id': 'req-8123ee4c-ebeb-4965-a560-fdff7a60ac85'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:11.174103 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-8123ee4c-ebeb-4965-a560-fdff7a60ac85 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '469a82384ecf4afaaeb572287fc49fb8', 'request_id': 'req-8123ee4c-ebeb-4965-a560-fdff7a60ac85'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:11.301374 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-8123ee4c-ebeb-4965-a560-fdff7a60ac85 admin admin] The token cache is being invalidate because project 469a82384ecf4afaaeb572287fc49fb8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80469) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 18 05:17:11.302046 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-8123ee4c-ebeb-4965-a560-fdff7a60ac85 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:11.302418 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-8123ee4c-ebeb-4965-a560-fdff7a60ac85 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:11.305201 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 969/1907] 10.222.0.23 () {58 vars in 1232 bytes} [Sat Jun 18 05:17:10 2022] DELETE /identity/v3/projects/469a82384ecf4afaaeb572287fc49fb8 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:17:11.310227 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d74feb9-5ef8-455b-bdaa-52fa0ea5555f None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:11.310802 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d74feb9-5ef8-455b-bdaa-52fa0ea5555f None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:11.311260 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d74feb9-5ef8-455b-bdaa-52fa0ea5555f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:11.312735 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:17:11.354030 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-9d74feb9-5ef8-455b-bdaa-52fa0ea5555f None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:17:11.355645 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-9d74feb9-5ef8-455b-bdaa-52fa0ea5555f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:17:11.446230 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 939/1908] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:17:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 18 05:17:11.455818 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381d17e8-cc95-471a-ab2c-882a634717c6 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:11.456205 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381d17e8-cc95-471a-ab2c-882a634717c6 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:11.456500 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381d17e8-cc95-471a-ab2c-882a634717c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:11.478508 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-381d17e8-cc95-471a-ab2c-882a634717c6 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:17:11.479724 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-381d17e8-cc95-471a-ab2c-882a634717c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:17:11.493581 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 970/1909] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:17:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:17:11.497464 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d74feb9-5ef8-455b-bdaa-52fa0ea5555f None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:11.514230 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381d17e8-cc95-471a-ab2c-882a634717c6 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:11.516235 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-381d17e8-cc95-471a-ab2c-882a634717c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:17:11.524400 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d74feb9-5ef8-455b-bdaa-52fa0ea5555f None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:11.533826 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381d17e8-cc95-471a-ab2c-882a634717c6 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:11.536499 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d508cdf-3268-4504-a55a-a154bb62ac52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:11.537276 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d508cdf-3268-4504-a55a-a154bb62ac52 admin admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:11.538806 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d508cdf-3268-4504-a55a-a154bb62ac52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:11.539111 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d508cdf-3268-4504-a55a-a154bb62ac52 admin admin] PATH_INFO: `/v3/domains` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:11.541257 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d508cdf-3268-4504-a55a-a154bb62ac52 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 05:17:11.541792 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d508cdf-3268-4504-a55a-a154bb62ac52 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:11.541956 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224ba06f-adad-4a10-a129-bb2002415ca6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:11.543475 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d508cdf-3268-4504-a55a-a154bb62ac52 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:11.544024 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224ba06f-adad-4a10-a129-bb2002415ca6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:11.544365 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224ba06f-adad-4a10-a129-bb2002415ca6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:11.544583 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224ba06f-adad-4a10-a129-bb2002415ca6 admin admin] PATH_INFO: `/v3/projects/3dfdbabe16354cd086783a2d6bc63fed` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:11.546114 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224ba06f-adad-4a10-a129-bb2002415ca6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3dfdbabe16354cd086783a2d6bc63fed)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:11.548772 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224ba06f-adad-4a10-a129-bb2002415ca6 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:11.589766 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 940/1910] 10.222.0.23 () {58 vars in 1152 bytes} [Sat Jun 18 05:17:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 05:17:11.595437 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d508cdf-3268-4504-a55a-a154bb62ac52 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:11.611143 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d508cdf-3268-4504-a55a-a154bb62ac52 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:11.627006 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d177d21d-1c6f-42f9-9100-62266cf44cd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:11.627247 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d177d21d-1c6f-42f9-9100-62266cf44cd5 admin admin] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:11.627538 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d177d21d-1c6f-42f9-9100-62266cf44cd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:11.627817 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d177d21d-1c6f-42f9-9100-62266cf44cd5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:11.628536 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d177d21d-1c6f-42f9-9100-62266cf44cd5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:11.630583 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d177d21d-1c6f-42f9-9100-62266cf44cd5 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:11.728598 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-224ba06f-adad-4a10-a129-bb2002415ca6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3dfdbabe16354cd086783a2d6bc63fed', 'request_id': 'req-224ba06f-adad-4a10-a129-bb2002415ca6'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:11.737750 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-224ba06f-adad-4a10-a129-bb2002415ca6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3dfdbabe16354cd086783a2d6bc63fed', 'request_id': 'req-224ba06f-adad-4a10-a129-bb2002415ca6'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:11.742028 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 941/1911] 10.222.0.23 () {60 vars in 1152 bytes} [Sat Jun 18 05:17:11 2022] POST /identity/v3/projects => generated 369 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:17:11.747204 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d177d21d-1c6f-42f9-9100-62266cf44cd5 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:11.749733 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-224ba06f-adad-4a10-a129-bb2002415ca6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3dfdbabe16354cd086783a2d6bc63fed', 'request_id': 'req-224ba06f-adad-4a10-a129-bb2002415ca6'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:11.814464 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-224ba06f-adad-4a10-a129-bb2002415ca6 admin admin] The token cache is being invalidate because project 3dfdbabe16354cd086783a2d6bc63fed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80469) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 18 05:17:11.814822 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-224ba06f-adad-4a10-a129-bb2002415ca6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:11.817900 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-224ba06f-adad-4a10-a129-bb2002415ca6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:11.822982 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 971/1912] 10.222.0.23 () {58 vars in 1232 bytes} [Sat Jun 18 05:17:11 2022] DELETE /identity/v3/projects/3dfdbabe16354cd086783a2d6bc63fed => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:17:11.824368 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224ba06f-adad-4a10-a129-bb2002415ca6 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:11.827188 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-224ba06f-adad-4a10-a129-bb2002415ca6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:17:11.830270 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d177d21d-1c6f-42f9-9100-62266cf44cd5 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:11.837874 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1095b3b1-efa9-4802-b240-6234f17681a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:11.839068 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1095b3b1-efa9-4802-b240-6234f17681a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:11.839464 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1095b3b1-efa9-4802-b240-6234f17681a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:11.840014 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1095b3b1-efa9-4802-b240-6234f17681a8 admin admin] PATH_INFO: `/v3/users` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:11.841560 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1095b3b1-efa9-4802-b240-6234f17681a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:11.844036 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1095b3b1-efa9-4802-b240-6234f17681a8 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:11.934659 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224ba06f-adad-4a10-a129-bb2002415ca6 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:11.939645 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 942/1913] 10.222.0.23 () {60 vars in 1143 bytes} [Sat Jun 18 05:17:11 2022] POST /identity/v3/users => generated 285 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 18 05:17:11.944363 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1095b3b1-efa9-4802-b240-6234f17681a8 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:11.946698 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-1095b3b1-efa9-4802-b240-6234f17681a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:17:11.953018 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f43d2dd-e612-48b7-97c0-32e224e8a678 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:11.954047 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f43d2dd-e612-48b7-97c0-32e224e8a678 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:11.954439 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f43d2dd-e612-48b7-97c0-32e224e8a678 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:11.955339 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f43d2dd-e612-48b7-97c0-32e224e8a678 admin admin] PATH_INFO: `/v3/users/baae48a07d7e417486294c655d88b5c0` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:11.956707 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f43d2dd-e612-48b7-97c0-32e224e8a678 admin admin] RBAC: Authorizing `identity:delete_user(user_id=baae48a07d7e417486294c655d88b5c0)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:11.959112 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f43d2dd-e612-48b7-97c0-32e224e8a678 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:11.961509 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1095b3b1-efa9-4802-b240-6234f17681a8 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:11.968331 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b97203-d46d-4b89-a7e9-585a349ae23f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:11.970703 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b97203-d46d-4b89-a7e9-585a349ae23f admin admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:11.971273 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b97203-d46d-4b89-a7e9-585a349ae23f admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:11.971641 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b97203-d46d-4b89-a7e9-585a349ae23f admin admin] PATH_INFO: `/v3/roles` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:11.972441 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b97203-d46d-4b89-a7e9-585a349ae23f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:11.974112 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b97203-d46d-4b89-a7e9-585a349ae23f admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:11.994244 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 943/1914] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:17:11 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:17:11.997991 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b97203-d46d-4b89-a7e9-585a349ae23f admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:12.012422 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b97203-d46d-4b89-a7e9-585a349ae23f admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:12.027555 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186017b8-197e-433e-bf94-29ae1bd00ade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:12.028554 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186017b8-197e-433e-bf94-29ae1bd00ade admin admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:12.029084 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186017b8-197e-433e-bf94-29ae1bd00ade admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:12.030748 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186017b8-197e-433e-bf94-29ae1bd00ade admin admin] PATH_INFO: `/v3/roles` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:12.031775 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186017b8-197e-433e-bf94-29ae1bd00ade admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:12.035818 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186017b8-197e-433e-bf94-29ae1bd00ade admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:12.053083 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 944/1915] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:17:11 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:17:12.054842 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186017b8-197e-433e-bf94-29ae1bd00ade admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:12.069731 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186017b8-197e-433e-bf94-29ae1bd00ade admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:12.074788 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c20d987-fdec-4ed5-a017-d44890628a8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:12.075823 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c20d987-fdec-4ed5-a017-d44890628a8d admin admin] REQUEST_METHOD: `PUT` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:12.076099 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c20d987-fdec-4ed5-a017-d44890628a8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:12.076403 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c20d987-fdec-4ed5-a017-d44890628a8d admin admin] PATH_INFO: `/v3/projects/1b37094738f44f49bb381c172488450f/users/d2368c56c4f448ea988d25f4f1325527/roles/cbf5c4c26898413d93e73b46b41046a3` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:12.092296 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c20d987-fdec-4ed5-a017-d44890628a8d admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b37094738f44f49bb381c172488450f, user_id=d2368c56c4f448ea988d25f4f1325527, role_id=cbf5c4c26898413d93e73b46b41046a3)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:12.095320 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c20d987-fdec-4ed5-a017-d44890628a8d admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:12.130609 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-0f43d2dd-e612-48b7-97c0-32e224e8a678 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'baae48a07d7e417486294c655d88b5c0'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:12.145101 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-0f43d2dd-e612-48b7-97c0-32e224e8a678 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'baae48a07d7e417486294c655d88b5c0'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:12.145871 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-0f43d2dd-e612-48b7-97c0-32e224e8a678 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'baae48a07d7e417486294c655d88b5c0'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:12.174606 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 945/1916] 10.222.0.23 () {60 vars in 1482 bytes} [Sat Jun 18 05:17:12 2022] PUT /identity/v3/projects/1b37094738f44f49bb381c172488450f/users/d2368c56c4f448ea988d25f4f1325527/roles/cbf5c4c26898413d93e73b46b41046a3 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:17:12.185922 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd4f220-3de0-4ffc-b052-90cfb91a80c4 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:12.186220 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd4f220-3de0-4ffc-b052-90cfb91a80c4 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:12.186506 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd4f220-3de0-4ffc-b052-90cfb91a80c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:12.218301 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-8fd4f220-3de0-4ffc-b052-90cfb91a80c4 None None] MFA Rules not processed for user `d2368c56c4f448ea988d25f4f1325527`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:17:12.226289 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-8fd4f220-3de0-4ffc-b052-90cfb91a80c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:17:12.236450 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-0f43d2dd-e612-48b7-97c0-32e224e8a678 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'baae48a07d7e417486294c655d88b5c0'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:12.246226 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-0f43d2dd-e612-48b7-97c0-32e224e8a678 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'baae48a07d7e417486294c655d88b5c0'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:12.254261 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 972/1917] 10.222.0.23 () {58 vars in 1223 bytes} [Sat Jun 18 05:17:11 2022] DELETE /identity/v3/users/baae48a07d7e417486294c655d88b5c0 => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:17:12.310505 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f43d2dd-e612-48b7-97c0-32e224e8a678 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:12.313229 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-0f43d2dd-e612-48b7-97c0-32e224e8a678 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:17:12.339441 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 946/1918] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:17:12 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 18 05:17:12.346672 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd4f220-3de0-4ffc-b052-90cfb91a80c4 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:12.351377 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-8fd4f220-3de0-4ffc-b052-90cfb91a80c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:17:12.393695 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f43d2dd-e612-48b7-97c0-32e224e8a678 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:12.398522 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f1fe5f-0cb9-417b-ae9e-8140067742e0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9de0428abc8400aa338aba4f7c34c94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:12.399449 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f1fe5f-0cb9-417b-ae9e-8140067742e0 service cinder] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:12.401677 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f1fe5f-0cb9-417b-ae9e-8140067742e0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:12.401677 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f1fe5f-0cb9-417b-ae9e-8140067742e0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:12.403827 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-01f1fe5f-0cb9-417b-ae9e-8140067742e0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:17:12.425490 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd4f220-3de0-4ffc-b052-90cfb91a80c4 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:12.470981 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f1fe5f-0cb9-417b-ae9e-8140067742e0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:12.472481 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f1fe5f-0cb9-417b-ae9e-8140067742e0 service cinder] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:12.483562 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fd5a6f-453b-4967-9d6a-6c4306110fda service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4638e5d1fa124667b07054a87c2d353e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:12.484524 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fd5a6f-453b-4967-9d6a-6c4306110fda service glance] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:12.484959 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fd5a6f-453b-4967-9d6a-6c4306110fda service glance] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:12.485725 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fd5a6f-453b-4967-9d6a-6c4306110fda service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:12.496755 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 973/1919] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:17:12 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 18 05:17:12.555359 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fd5a6f-453b-4967-9d6a-6c4306110fda service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:12.557656 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fd5a6f-453b-4967-9d6a-6c4306110fda service glance] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:12.571563 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 947/1920] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:17:12 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:17:12.758678 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:17:12.907555 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f1fe5f-0cb9-417b-ae9e-8140067742e0 service cinder] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:12.909289 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-01f1fe5f-0cb9-417b-ae9e-8140067742e0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:17:12.983504 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f1fe5f-0cb9-417b-ae9e-8140067742e0 service cinder] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:12.988226 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0372dfda-7e16-47ac-8dd0-e4158a5e3b41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:12.989262 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0372dfda-7e16-47ac-8dd0-e4158a5e3b41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:12.989785 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0372dfda-7e16-47ac-8dd0-e4158a5e3b41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:12.989982 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0372dfda-7e16-47ac-8dd0-e4158a5e3b41 admin admin] PATH_INFO: `/v3/projects/f6485bcce7fb40139f08f61755d773f8` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:12.991306 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0372dfda-7e16-47ac-8dd0-e4158a5e3b41 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f6485bcce7fb40139f08f61755d773f8)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:12.993523 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0372dfda-7e16-47ac-8dd0-e4158a5e3b41 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:13.125229 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-0372dfda-7e16-47ac-8dd0-e4158a5e3b41 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f6485bcce7fb40139f08f61755d773f8', 'request_id': 'req-0372dfda-7e16-47ac-8dd0-e4158a5e3b41'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:13.130639 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-0372dfda-7e16-47ac-8dd0-e4158a5e3b41 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f6485bcce7fb40139f08f61755d773f8', 'request_id': 'req-0372dfda-7e16-47ac-8dd0-e4158a5e3b41'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:13.137023 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-0372dfda-7e16-47ac-8dd0-e4158a5e3b41 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f6485bcce7fb40139f08f61755d773f8', 'request_id': 'req-0372dfda-7e16-47ac-8dd0-e4158a5e3b41'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:13.231819 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-0372dfda-7e16-47ac-8dd0-e4158a5e3b41 admin admin] The token cache is being invalidate because project f6485bcce7fb40139f08f61755d773f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80469) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 18 05:17:13.232111 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-0372dfda-7e16-47ac-8dd0-e4158a5e3b41 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:13.232903 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-0372dfda-7e16-47ac-8dd0-e4158a5e3b41 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:13.234624 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 974/1921] 10.222.0.23 () {58 vars in 1232 bytes} [Sat Jun 18 05:17:12 2022] DELETE /identity/v3/projects/f6485bcce7fb40139f08f61755d773f8 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:17:13.239522 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992ccbb5-8af3-431c-9a4b-f1990aeeac57 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:13.239810 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992ccbb5-8af3-431c-9a4b-f1990aeeac57 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:13.240082 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992ccbb5-8af3-431c-9a4b-f1990aeeac57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:13.261056 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-992ccbb5-8af3-431c-9a4b-f1990aeeac57 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:17:13.262301 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-992ccbb5-8af3-431c-9a4b-f1990aeeac57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:17:13.325009 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 948/1922] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:17:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:17:13.334016 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edffc786-774f-44dd-b549-f4ac83a74803 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:13.334684 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edffc786-774f-44dd-b549-f4ac83a74803 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:13.334818 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edffc786-774f-44dd-b549-f4ac83a74803 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:13.354157 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-edffc786-774f-44dd-b549-f4ac83a74803 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:17:13.355378 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-edffc786-774f-44dd-b549-f4ac83a74803 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:17:13.363325 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 975/1923] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:17:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:17:13.368023 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992ccbb5-8af3-431c-9a4b-f1990aeeac57 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:13.377773 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992ccbb5-8af3-431c-9a4b-f1990aeeac57 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:13.383058 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68024e6e-f2dd-40fe-a536-88c102038608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:13.383722 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68024e6e-f2dd-40fe-a536-88c102038608 admin admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:13.384007 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68024e6e-f2dd-40fe-a536-88c102038608 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:13.384286 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68024e6e-f2dd-40fe-a536-88c102038608 admin admin] PATH_INFO: `/v3/domains` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:13.385015 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68024e6e-f2dd-40fe-a536-88c102038608 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 05:17:13.385329 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68024e6e-f2dd-40fe-a536-88c102038608 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:13.386966 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68024e6e-f2dd-40fe-a536-88c102038608 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:13.398792 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 949/1924] 10.222.0.23 () {58 vars in 1152 bytes} [Sat Jun 18 05:17:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 05:17:13.403357 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edffc786-774f-44dd-b549-f4ac83a74803 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:13.412667 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edffc786-774f-44dd-b549-f4ac83a74803 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:13.417491 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0358cf0-6474-4382-94cb-9da71d973168 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:13.418135 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0358cf0-6474-4382-94cb-9da71d973168 admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:13.418417 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0358cf0-6474-4382-94cb-9da71d973168 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:13.418694 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0358cf0-6474-4382-94cb-9da71d973168 admin admin] PATH_INFO: `/v3/projects` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:13.419391 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0358cf0-6474-4382-94cb-9da71d973168 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:13.421318 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0358cf0-6474-4382-94cb-9da71d973168 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:13.540345 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 976/1925] 10.222.0.23 () {60 vars in 1152 bytes} [Sat Jun 18 05:17:13 2022] POST /identity/v3/projects => generated 371 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:17:13.544483 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68024e6e-f2dd-40fe-a536-88c102038608 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:13.554535 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68024e6e-f2dd-40fe-a536-88c102038608 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:13.612548 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09ef761-d9c4-478d-9a1a-90dc1b4a6ef6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:13.613219 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09ef761-d9c4-478d-9a1a-90dc1b4a6ef6 admin admin] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:13.613530 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09ef761-d9c4-478d-9a1a-90dc1b4a6ef6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:13.613809 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09ef761-d9c4-478d-9a1a-90dc1b4a6ef6 admin admin] PATH_INFO: `/v3/users` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:13.614502 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09ef761-d9c4-478d-9a1a-90dc1b4a6ef6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:13.617025 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09ef761-d9c4-478d-9a1a-90dc1b4a6ef6 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:13.630328 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:17:13.684908 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 950/1926] 10.222.0.23 () {60 vars in 1143 bytes} [Sat Jun 18 05:17:13 2022] POST /identity/v3/users => generated 286 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:17:13.686108 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0358cf0-6474-4382-94cb-9da71d973168 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:13.697097 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0358cf0-6474-4382-94cb-9da71d973168 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:13.701815 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463b3c34-3b9e-4eec-893b-accae99ef725 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:13.702469 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463b3c34-3b9e-4eec-893b-accae99ef725 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:13.702984 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463b3c34-3b9e-4eec-893b-accae99ef725 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:13.705709 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463b3c34-3b9e-4eec-893b-accae99ef725 admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:13.706500 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463b3c34-3b9e-4eec-893b-accae99ef725 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:13.715235 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463b3c34-3b9e-4eec-893b-accae99ef725 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:13.724871 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 977/1927] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:17:13 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:17:13.728870 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09ef761-d9c4-478d-9a1a-90dc1b4a6ef6 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:13.738901 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09ef761-d9c4-478d-9a1a-90dc1b4a6ef6 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:13.743649 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e125a9-5f71-4d86-863e-cd25018d822f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:13.744301 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e125a9-5f71-4d86-863e-cd25018d822f admin admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:13.744586 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e125a9-5f71-4d86-863e-cd25018d822f admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:13.744859 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e125a9-5f71-4d86-863e-cd25018d822f admin admin] PATH_INFO: `/v3/roles` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:13.745653 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e125a9-5f71-4d86-863e-cd25018d822f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:13.747312 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e125a9-5f71-4d86-863e-cd25018d822f admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:13.759350 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 951/1928] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:17:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:17:13.762744 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463b3c34-3b9e-4eec-893b-accae99ef725 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:13.772831 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463b3c34-3b9e-4eec-893b-accae99ef725 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:13.778021 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b59bdc-90f0-4720-8683-2a3c6457100f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:13.778904 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b59bdc-90f0-4720-8683-2a3c6457100f admin admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:13.779183 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b59bdc-90f0-4720-8683-2a3c6457100f admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:13.779554 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b59bdc-90f0-4720-8683-2a3c6457100f admin admin] PATH_INFO: `/v3/projects/a77c57e4cfe14492b5622493087c3200/users/99364ba0687443d893e2b3c1a1756ee4/roles/cbf5c4c26898413d93e73b46b41046a3` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:13.795174 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b59bdc-90f0-4720-8683-2a3c6457100f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a77c57e4cfe14492b5622493087c3200, user_id=99364ba0687443d893e2b3c1a1756ee4, role_id=cbf5c4c26898413d93e73b46b41046a3)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:13.797907 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b59bdc-90f0-4720-8683-2a3c6457100f admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:13.885000 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 978/1929] 10.222.0.23 () {60 vars in 1482 bytes} [Sat Jun 18 05:17:13 2022] PUT /identity/v3/projects/a77c57e4cfe14492b5622493087c3200/users/99364ba0687443d893e2b3c1a1756ee4/roles/cbf5c4c26898413d93e73b46b41046a3 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:17:13.894990 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b17a6d4-dee6-4105-b6eb-3935f99101c2 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:13.895383 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b17a6d4-dee6-4105-b6eb-3935f99101c2 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:13.895768 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b17a6d4-dee6-4105-b6eb-3935f99101c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:13.925095 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-7b17a6d4-dee6-4105-b6eb-3935f99101c2 None None] MFA Rules not processed for user `99364ba0687443d893e2b3c1a1756ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:17:13.926404 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-7b17a6d4-dee6-4105-b6eb-3935f99101c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:17:13.999405 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 952/1930] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:17:13 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:17:14.003624 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b59bdc-90f0-4720-8683-2a3c6457100f admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:14.019132 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:17:14.072790 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b59bdc-90f0-4720-8683-2a3c6457100f admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:14.077662 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555b74fa-9d5b-45c5-a86f-a7832c2b4561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:14.078314 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555b74fa-9d5b-45c5-a86f-a7832c2b4561 admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:14.078592 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555b74fa-9d5b-45c5-a86f-a7832c2b4561 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:14.078861 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555b74fa-9d5b-45c5-a86f-a7832c2b4561 admin admin] PATH_INFO: `/v3/projects` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:14.079617 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555b74fa-9d5b-45c5-a86f-a7832c2b4561 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:14.081550 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555b74fa-9d5b-45c5-a86f-a7832c2b4561 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:14.110354 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:17:14.170153 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 979/1931] 10.222.0.23 () {60 vars in 1152 bytes} [Sat Jun 18 05:17:14 2022] POST /identity/v3/projects => generated 373 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 18 05:17:14.174057 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b17a6d4-dee6-4105-b6eb-3935f99101c2 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:14.184982 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b17a6d4-dee6-4105-b6eb-3935f99101c2 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:14.241502 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106fb963-be83-4f98-96c6-f3e3196efbc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:14.242402 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106fb963-be83-4f98-96c6-f3e3196efbc7 admin admin] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:14.242861 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106fb963-be83-4f98-96c6-f3e3196efbc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:14.243351 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106fb963-be83-4f98-96c6-f3e3196efbc7 admin admin] PATH_INFO: `/v3/users` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:14.244629 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106fb963-be83-4f98-96c6-f3e3196efbc7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:14.248252 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106fb963-be83-4f98-96c6-f3e3196efbc7 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:14.362190 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 953/1932] 10.222.0.23 () {60 vars in 1143 bytes} [Sat Jun 18 05:17:14 2022] POST /identity/v3/users => generated 293 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:17:14.368428 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555b74fa-9d5b-45c5-a86f-a7832c2b4561 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:14.379287 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555b74fa-9d5b-45c5-a86f-a7832c2b4561 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:14.385786 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5aa78f-78da-4b62-87ed-6b1012a82093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:14.392092 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5aa78f-78da-4b62-87ed-6b1012a82093 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:14.392092 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5aa78f-78da-4b62-87ed-6b1012a82093 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:14.392092 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5aa78f-78da-4b62-87ed-6b1012a82093 admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:14.392092 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5aa78f-78da-4b62-87ed-6b1012a82093 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:14.392092 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5aa78f-78da-4b62-87ed-6b1012a82093 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:14.405594 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 980/1933] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:17:14 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:17:14.406799 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106fb963-be83-4f98-96c6-f3e3196efbc7 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:14.418549 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106fb963-be83-4f98-96c6-f3e3196efbc7 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:14.423316 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec42d985-29d5-428e-8fa4-53635c401a43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:14.424203 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec42d985-29d5-428e-8fa4-53635c401a43 admin admin] REQUEST_METHOD: `PUT` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:14.424485 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec42d985-29d5-428e-8fa4-53635c401a43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:14.424758 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec42d985-29d5-428e-8fa4-53635c401a43 admin admin] PATH_INFO: `/v3/projects/6b0b979bf0794047b7bba224428a6186/users/2cd9afd1973b4da29ce0b26a2c0168bd/roles/21c65f2c51bd4dd283d13507d2714467` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:14.440963 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec42d985-29d5-428e-8fa4-53635c401a43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b0b979bf0794047b7bba224428a6186, user_id=2cd9afd1973b4da29ce0b26a2c0168bd, role_id=21c65f2c51bd4dd283d13507d2714467)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:14.446874 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec42d985-29d5-428e-8fa4-53635c401a43 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:14.494614 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 954/1934] 10.222.0.23 () {60 vars in 1482 bytes} [Sat Jun 18 05:17:14 2022] PUT /identity/v3/projects/6b0b979bf0794047b7bba224428a6186/users/2cd9afd1973b4da29ce0b26a2c0168bd/roles/21c65f2c51bd4dd283d13507d2714467 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:17:14.498469 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5aa78f-78da-4b62-87ed-6b1012a82093 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:14.512019 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5aa78f-78da-4b62-87ed-6b1012a82093 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:14.582625 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09cf6c6-5383-40e6-ad06-1b319c543e6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:14.583365 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09cf6c6-5383-40e6-ad06-1b319c543e6f admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:14.583648 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09cf6c6-5383-40e6-ad06-1b319c543e6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:14.583965 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09cf6c6-5383-40e6-ad06-1b319c543e6f admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:14.585308 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09cf6c6-5383-40e6-ad06-1b319c543e6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:14.587051 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09cf6c6-5383-40e6-ad06-1b319c543e6f admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:14.597475 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 981/1935] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:17:14 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:17:14.601825 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec42d985-29d5-428e-8fa4-53635c401a43 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:14.612171 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec42d985-29d5-428e-8fa4-53635c401a43 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:14.618746 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f41610-0abd-4b57-8b38-7f493078b71c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:14.618746 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f41610-0abd-4b57-8b38-7f493078b71c admin admin] REQUEST_METHOD: `PUT` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:14.618746 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f41610-0abd-4b57-8b38-7f493078b71c admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:14.618746 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f41610-0abd-4b57-8b38-7f493078b71c admin admin] PATH_INFO: `/v3/projects/6b0b979bf0794047b7bba224428a6186/users/2cd9afd1973b4da29ce0b26a2c0168bd/roles/21c65f2c51bd4dd283d13507d2714467` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:14.620789 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f41610-0abd-4b57-8b38-7f493078b71c admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b0b979bf0794047b7bba224428a6186, user_id=2cd9afd1973b4da29ce0b26a2c0168bd, role_id=21c65f2c51bd4dd283d13507d2714467)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:14.623499 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f41610-0abd-4b57-8b38-7f493078b71c admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:14.634920 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 955/1936] 10.222.0.23 () {60 vars in 1482 bytes} [Sat Jun 18 05:17:14 2022] PUT /identity/v3/projects/6b0b979bf0794047b7bba224428a6186/users/2cd9afd1973b4da29ce0b26a2c0168bd/roles/21c65f2c51bd4dd283d13507d2714467 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:17:14.645337 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee60f6e-f501-4c38-8560-c86807b3b51a None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:14.645663 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee60f6e-f501-4c38-8560-c86807b3b51a None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:14.645946 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee60f6e-f501-4c38-8560-c86807b3b51a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:14.673074 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-dee60f6e-f501-4c38-8560-c86807b3b51a None None] MFA Rules not processed for user `2cd9afd1973b4da29ce0b26a2c0168bd`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:17:14.674317 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-dee60f6e-f501-4c38-8560-c86807b3b51a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:17:14.753573 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 982/1937] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:17:14 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:17:14.763001 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f41610-0abd-4b57-8b38-7f493078b71c admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:14.765298 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-70f41610-0abd-4b57-8b38-7f493078b71c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:17:14.825803 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f41610-0abd-4b57-8b38-7f493078b71c admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:14.829885 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfb78eb-7f4b-48ef-abb3-dfa08295c0a1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9de0428abc8400aa338aba4f7c34c94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:14.830524 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfb78eb-7f4b-48ef-abb3-dfa08295c0a1 service cinder] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:14.830807 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfb78eb-7f4b-48ef-abb3-dfa08295c0a1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:14.831081 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfb78eb-7f4b-48ef-abb3-dfa08295c0a1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:14.841518 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfb78eb-7f4b-48ef-abb3-dfa08295c0a1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:14.843125 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfb78eb-7f4b-48ef-abb3-dfa08295c0a1 service cinder] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:14.854628 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 956/1938] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:17:14 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:17:15.168324 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:17:15.299664 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:17:15.808506 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee60f6e-f501-4c38-8560-c86807b3b51a None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:15.876815 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee60f6e-f501-4c38-8560-c86807b3b51a None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:15.881441 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3155ca47-0a51-4d20-97bf-b9db066749f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:15.882925 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3155ca47-0a51-4d20-97bf-b9db066749f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:15.883205 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3155ca47-0a51-4d20-97bf-b9db066749f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:15.883534 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3155ca47-0a51-4d20-97bf-b9db066749f2 admin admin] PATH_INFO: `/v3/users/99364ba0687443d893e2b3c1a1756ee4` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:15.884775 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3155ca47-0a51-4d20-97bf-b9db066749f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=99364ba0687443d893e2b3c1a1756ee4)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:15.886845 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3155ca47-0a51-4d20-97bf-b9db066749f2 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:16.067569 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-3155ca47-0a51-4d20-97bf-b9db066749f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '99364ba0687443d893e2b3c1a1756ee4'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:16.080648 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-3155ca47-0a51-4d20-97bf-b9db066749f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '99364ba0687443d893e2b3c1a1756ee4'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:16.088400 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-3155ca47-0a51-4d20-97bf-b9db066749f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '99364ba0687443d893e2b3c1a1756ee4'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:16.133037 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:17:16.184712 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-3155ca47-0a51-4d20-97bf-b9db066749f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '99364ba0687443d893e2b3c1a1756ee4'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:16.193881 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-3155ca47-0a51-4d20-97bf-b9db066749f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '99364ba0687443d893e2b3c1a1756ee4'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:16.202607 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfb78eb-7f4b-48ef-abb3-dfa08295c0a1 service cinder] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:16.203777 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-7dfb78eb-7f4b-48ef-abb3-dfa08295c0a1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:17:16.211106 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 983/1939] 10.222.0.23 () {58 vars in 1223 bytes} [Sat Jun 18 05:17:15 2022] DELETE /identity/v3/users/99364ba0687443d893e2b3c1a1756ee4 => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:17:16.214111 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58560eb5-998e-4c3b-a744-517350766e90 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:16.214517 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58560eb5-998e-4c3b-a744-517350766e90 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:16.214853 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58560eb5-998e-4c3b-a744-517350766e90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:16.224458 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfb78eb-7f4b-48ef-abb3-dfa08295c0a1 service cinder] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:16.239190 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-58560eb5-998e-4c3b-a744-517350766e90 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:17:16.241261 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-58560eb5-998e-4c3b-a744-517350766e90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:17:16.301181 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fcadb8-a2de-4998-81d9-c31517d8c387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:16.302194 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fcadb8-a2de-4998-81d9-c31517d8c387 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:16.302479 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fcadb8-a2de-4998-81d9-c31517d8c387 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:16.302756 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fcadb8-a2de-4998-81d9-c31517d8c387 admin admin] PATH_INFO: `/v3/users/fcf611fb28de427a91d438c9af2ed4c2` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:16.304845 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fcadb8-a2de-4998-81d9-c31517d8c387 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fcf611fb28de427a91d438c9af2ed4c2)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:16.306836 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fcadb8-a2de-4998-81d9-c31517d8c387 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:16.334071 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 984/1940] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:17:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:17:16.346360 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58560eb5-998e-4c3b-a744-517350766e90 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:16.348180 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-58560eb5-998e-4c3b-a744-517350766e90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:17:16.429593 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58560eb5-998e-4c3b-a744-517350766e90 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:16.434151 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e073d5-df21-498e-846e-de68e1e62ace service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c4d7cde8f4ad9aa479bb09e325db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:16.434829 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e073d5-df21-498e-846e-de68e1e62ace service neutron] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:16.440109 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e073d5-df21-498e-846e-de68e1e62ace service neutron] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:16.440109 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e073d5-df21-498e-846e-de68e1e62ace service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:16.446774 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e073d5-df21-498e-846e-de68e1e62ace service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:16.448268 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e073d5-df21-498e-846e-de68e1e62ace service neutron] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:16.462363 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 985/1941] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:17:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 18 05:17:16.515507 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-93fcadb8-a2de-4998-81d9-c31517d8c387 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fcf611fb28de427a91d438c9af2ed4c2'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:16.517175 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-93fcadb8-a2de-4998-81d9-c31517d8c387 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fcf611fb28de427a91d438c9af2ed4c2'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:16.642590 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-93fcadb8-a2de-4998-81d9-c31517d8c387 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fcf611fb28de427a91d438c9af2ed4c2'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:16.663970 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-93fcadb8-a2de-4998-81d9-c31517d8c387 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fcf611fb28de427a91d438c9af2ed4c2'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:16.673644 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-93fcadb8-a2de-4998-81d9-c31517d8c387 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fcf611fb28de427a91d438c9af2ed4c2'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:16.681085 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 957/1942] 10.222.0.23 () {58 vars in 1223 bytes} [Sat Jun 18 05:17:16 2022] DELETE /identity/v3/users/fcf611fb28de427a91d438c9af2ed4c2 => generated 0 bytes in 479 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 18 05:17:16.685917 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb494ed3-6591-424e-b9cf-b21e955068e2 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:16.687106 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb494ed3-6591-424e-b9cf-b21e955068e2 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:16.687252 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb494ed3-6591-424e-b9cf-b21e955068e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:16.711102 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-eb494ed3-6591-424e-b9cf-b21e955068e2 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:17:16.712242 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-eb494ed3-6591-424e-b9cf-b21e955068e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:17:16.798077 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 986/1943] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:17:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:17:16.811802 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fcadb8-a2de-4998-81d9-c31517d8c387 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:16.813664 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-93fcadb8-a2de-4998-81d9-c31517d8c387 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:17:16.905357 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fcadb8-a2de-4998-81d9-c31517d8c387 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:16.909552 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40ef7bf-1df4-41aa-9024-c2a5287d28ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c4d7cde8f4ad9aa479bb09e325db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:16.910223 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40ef7bf-1df4-41aa-9024-c2a5287d28ef service neutron] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:16.910591 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40ef7bf-1df4-41aa-9024-c2a5287d28ef service neutron] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:16.910915 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40ef7bf-1df4-41aa-9024-c2a5287d28ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:16.920956 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40ef7bf-1df4-41aa-9024-c2a5287d28ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:16.922593 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40ef7bf-1df4-41aa-9024-c2a5287d28ef service neutron] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:16.934068 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 958/1944] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:17:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:17:16.974504 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb494ed3-6591-424e-b9cf-b21e955068e2 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:16.976110 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-eb494ed3-6591-424e-b9cf-b21e955068e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:17:17.014321 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb494ed3-6591-424e-b9cf-b21e955068e2 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:17.031083 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574f50be-d784-4c7c-90b1-8ad51a9efe4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:17.031083 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574f50be-d784-4c7c-90b1-8ad51a9efe4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:17.031083 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574f50be-d784-4c7c-90b1-8ad51a9efe4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:17.031647 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574f50be-d784-4c7c-90b1-8ad51a9efe4a admin admin] PATH_INFO: `/v3/projects/a77c57e4cfe14492b5622493087c3200` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:17.033679 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574f50be-d784-4c7c-90b1-8ad51a9efe4a admin admin] RBAC: Authorizing `identity:delete_project(project_id=a77c57e4cfe14492b5622493087c3200)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:17.039095 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574f50be-d784-4c7c-90b1-8ad51a9efe4a admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:17.226740 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-574f50be-d784-4c7c-90b1-8ad51a9efe4a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a77c57e4cfe14492b5622493087c3200', 'request_id': 'req-574f50be-d784-4c7c-90b1-8ad51a9efe4a'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:17.230680 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:17:17.234643 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-574f50be-d784-4c7c-90b1-8ad51a9efe4a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a77c57e4cfe14492b5622493087c3200', 'request_id': 'req-574f50be-d784-4c7c-90b1-8ad51a9efe4a'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:17.241626 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-574f50be-d784-4c7c-90b1-8ad51a9efe4a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a77c57e4cfe14492b5622493087c3200', 'request_id': 'req-574f50be-d784-4c7c-90b1-8ad51a9efe4a'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:17.321117 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-574f50be-d784-4c7c-90b1-8ad51a9efe4a admin admin] The token cache is being invalidate because project a77c57e4cfe14492b5622493087c3200 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80469) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 18 05:17:17.321508 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-574f50be-d784-4c7c-90b1-8ad51a9efe4a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:17.322174 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-574f50be-d784-4c7c-90b1-8ad51a9efe4a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:17.328610 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40ef7bf-1df4-41aa-9024-c2a5287d28ef service neutron] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:17.329299 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 987/1945] 10.222.0.23 () {58 vars in 1232 bytes} [Sat Jun 18 05:17:16 2022] DELETE /identity/v3/projects/a77c57e4cfe14492b5622493087c3200 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:17:17.349233 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40ef7bf-1df4-41aa-9024-c2a5287d28ef service neutron] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:17.426028 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04df0a8-fb64-4803-8fbe-ffa8dea4ccce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:17.427405 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04df0a8-fb64-4803-8fbe-ffa8dea4ccce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:17.427835 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04df0a8-fb64-4803-8fbe-ffa8dea4ccce admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:17.428405 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04df0a8-fb64-4803-8fbe-ffa8dea4ccce admin admin] PATH_INFO: `/v3/users/2cd9afd1973b4da29ce0b26a2c0168bd` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:17.430810 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04df0a8-fb64-4803-8fbe-ffa8dea4ccce admin admin] RBAC: Authorizing `identity:delete_user(user_id=2cd9afd1973b4da29ce0b26a2c0168bd)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:17.432997 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04df0a8-fb64-4803-8fbe-ffa8dea4ccce admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:17.607856 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-d04df0a8-fb64-4803-8fbe-ffa8dea4ccce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2cd9afd1973b4da29ce0b26a2c0168bd'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:17.608848 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-d04df0a8-fb64-4803-8fbe-ffa8dea4ccce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2cd9afd1973b4da29ce0b26a2c0168bd'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:17.628473 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574f50be-d784-4c7c-90b1-8ad51a9efe4a admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:17.628473 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-574f50be-d784-4c7c-90b1-8ad51a9efe4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:17:17.644790 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574f50be-d784-4c7c-90b1-8ad51a9efe4a admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:17.654560 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199ddf82-3b57-432d-aa24-4b87d0f9f160 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:17.654560 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199ddf82-3b57-432d-aa24-4b87d0f9f160 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:17.654560 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199ddf82-3b57-432d-aa24-4b87d0f9f160 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:17.654560 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199ddf82-3b57-432d-aa24-4b87d0f9f160 admin admin] PATH_INFO: `/v3/projects/b471c5c09317491eb8a3ac7f954ee544` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:17.655686 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199ddf82-3b57-432d-aa24-4b87d0f9f160 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b471c5c09317491eb8a3ac7f954ee544)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:17.657680 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199ddf82-3b57-432d-aa24-4b87d0f9f160 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:17.715641 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-d04df0a8-fb64-4803-8fbe-ffa8dea4ccce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2cd9afd1973b4da29ce0b26a2c0168bd'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:17.728836 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-d04df0a8-fb64-4803-8fbe-ffa8dea4ccce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2cd9afd1973b4da29ce0b26a2c0168bd'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:17.735531 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-d04df0a8-fb64-4803-8fbe-ffa8dea4ccce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2cd9afd1973b4da29ce0b26a2c0168bd'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:17.741478 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 959/1946] 10.222.0.23 () {58 vars in 1223 bytes} [Sat Jun 18 05:17:17 2022] DELETE /identity/v3/users/2cd9afd1973b4da29ce0b26a2c0168bd => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:17:17.847451 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-199ddf82-3b57-432d-aa24-4b87d0f9f160 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b471c5c09317491eb8a3ac7f954ee544', 'request_id': 'req-199ddf82-3b57-432d-aa24-4b87d0f9f160'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:17.854920 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-199ddf82-3b57-432d-aa24-4b87d0f9f160 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b471c5c09317491eb8a3ac7f954ee544', 'request_id': 'req-199ddf82-3b57-432d-aa24-4b87d0f9f160'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:17.863211 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-199ddf82-3b57-432d-aa24-4b87d0f9f160 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b471c5c09317491eb8a3ac7f954ee544', 'request_id': 'req-199ddf82-3b57-432d-aa24-4b87d0f9f160'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:17.974901 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-199ddf82-3b57-432d-aa24-4b87d0f9f160 admin admin] The token cache is being invalidate because project b471c5c09317491eb8a3ac7f954ee544 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80469) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 18 05:17:17.976407 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-199ddf82-3b57-432d-aa24-4b87d0f9f160 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:17.977244 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-199ddf82-3b57-432d-aa24-4b87d0f9f160 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:17.979259 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 988/1947] 10.222.0.23 () {58 vars in 1232 bytes} [Sat Jun 18 05:17:17 2022] DELETE /identity/v3/projects/b471c5c09317491eb8a3ac7f954ee544 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 18 05:17:17.984973 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48717441-1730-4219-aba7-25d29964d5c4 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:17.985553 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48717441-1730-4219-aba7-25d29964d5c4 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:17.985958 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48717441-1730-4219-aba7-25d29964d5c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:18.008272 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-48717441-1730-4219-aba7-25d29964d5c4 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:17:18.010079 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-48717441-1730-4219-aba7-25d29964d5c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:17:18.079288 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 960/1948] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:17:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 18 05:17:18.087760 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41a8af7-df07-4e66-abfe-dccc25d33d9e None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:18.088148 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41a8af7-df07-4e66-abfe-dccc25d33d9e None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:18.088549 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41a8af7-df07-4e66-abfe-dccc25d33d9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:18.141022 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-a41a8af7-df07-4e66-abfe-dccc25d33d9e None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:17:18.141022 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-a41a8af7-df07-4e66-abfe-dccc25d33d9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:17:18.157181 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 989/1949] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:17:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:17:18.161141 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48717441-1730-4219-aba7-25d29964d5c4 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:18.174484 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48717441-1730-4219-aba7-25d29964d5c4 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:18.188376 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31769b8b-9695-48f2-acd5-06cf465bfe55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:18.190089 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31769b8b-9695-48f2-acd5-06cf465bfe55 admin admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:18.190384 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31769b8b-9695-48f2-acd5-06cf465bfe55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:18.190683 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31769b8b-9695-48f2-acd5-06cf465bfe55 admin admin] PATH_INFO: `/v3/domains` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:18.191704 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31769b8b-9695-48f2-acd5-06cf465bfe55 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 05:17:18.192077 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31769b8b-9695-48f2-acd5-06cf465bfe55 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:18.205691 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31769b8b-9695-48f2-acd5-06cf465bfe55 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:18.233019 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 961/1950] 10.222.0.23 () {58 vars in 1152 bytes} [Sat Jun 18 05:17:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 05:17:18.233522 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41a8af7-df07-4e66-abfe-dccc25d33d9e None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:18.241310 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41a8af7-df07-4e66-abfe-dccc25d33d9e None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:18.247269 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea905a4-e84b-467e-b1cd-d967cd0894a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:18.247805 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea905a4-e84b-467e-b1cd-d967cd0894a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:18.248900 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea905a4-e84b-467e-b1cd-d967cd0894a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:18.249193 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea905a4-e84b-467e-b1cd-d967cd0894a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:18.250047 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea905a4-e84b-467e-b1cd-d967cd0894a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:18.253124 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea905a4-e84b-467e-b1cd-d967cd0894a2 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:18.383114 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31769b8b-9695-48f2-acd5-06cf465bfe55 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:18.391262 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-31769b8b-9695-48f2-acd5-06cf465bfe55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:17:18.415230 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31769b8b-9695-48f2-acd5-06cf465bfe55 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:18.418379 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 990/1951] 10.222.0.23 () {60 vars in 1152 bytes} [Sat Jun 18 05:17:18 2022] POST /identity/v3/projects => generated 373 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 18 05:17:18.422175 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea905a4-e84b-467e-b1cd-d967cd0894a2 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:18.514422 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9cede6-83ba-4dfc-88b5-76d7065b4c9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:18.514422 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9cede6-83ba-4dfc-88b5-76d7065b4c9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:18.514422 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9cede6-83ba-4dfc-88b5-76d7065b4c9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:18.514422 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9cede6-83ba-4dfc-88b5-76d7065b4c9a admin admin] PATH_INFO: `/v3/projects/6b0b979bf0794047b7bba224428a6186` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:18.514897 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9cede6-83ba-4dfc-88b5-76d7065b4c9a admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b0b979bf0794047b7bba224428a6186)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:18.519183 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9cede6-83ba-4dfc-88b5-76d7065b4c9a admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:18.602221 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea905a4-e84b-467e-b1cd-d967cd0894a2 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:18.608652 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de0e6dc-249a-42d3-b9ab-53a76c4367bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:18.609761 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de0e6dc-249a-42d3-b9ab-53a76c4367bf admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:18.610146 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de0e6dc-249a-42d3-b9ab-53a76c4367bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:18.610526 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de0e6dc-249a-42d3-b9ab-53a76c4367bf admin admin] PATH_INFO: `/v3/users` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:18.611497 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de0e6dc-249a-42d3-b9ab-53a76c4367bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:18.619983 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de0e6dc-249a-42d3-b9ab-53a76c4367bf admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:18.635119 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-8f9cede6-83ba-4dfc-88b5-76d7065b4c9a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b0b979bf0794047b7bba224428a6186', 'request_id': 'req-8f9cede6-83ba-4dfc-88b5-76d7065b4c9a'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:18.640797 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-8f9cede6-83ba-4dfc-88b5-76d7065b4c9a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b0b979bf0794047b7bba224428a6186', 'request_id': 'req-8f9cede6-83ba-4dfc-88b5-76d7065b4c9a'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:18.728898 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-8f9cede6-83ba-4dfc-88b5-76d7065b4c9a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b0b979bf0794047b7bba224428a6186', 'request_id': 'req-8f9cede6-83ba-4dfc-88b5-76d7065b4c9a'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:18.736749 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 991/1952] 10.222.0.23 () {60 vars in 1143 bytes} [Sat Jun 18 05:17:18 2022] POST /identity/v3/users => generated 287 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:17:18.738043 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de0e6dc-249a-42d3-b9ab-53a76c4367bf admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:18.752352 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-8f9cede6-83ba-4dfc-88b5-76d7065b4c9a admin admin] The token cache is being invalidate because project 6b0b979bf0794047b7bba224428a6186 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80468) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 18 05:17:18.755813 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-8f9cede6-83ba-4dfc-88b5-76d7065b4c9a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:18.755931 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de0e6dc-249a-42d3-b9ab-53a76c4367bf admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:18.756013 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-8f9cede6-83ba-4dfc-88b5-76d7065b4c9a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:18.756013 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 962/1953] 10.222.0.23 () {58 vars in 1232 bytes} [Sat Jun 18 05:17:18 2022] DELETE /identity/v3/projects/6b0b979bf0794047b7bba224428a6186 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:17:18.761525 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e027d390-a60e-4695-a9ab-cab3dbfa9d39 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:18.762020 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e027d390-a60e-4695-a9ab-cab3dbfa9d39 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:18.762487 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e027d390-a60e-4695-a9ab-cab3dbfa9d39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:18.804356 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:17:18.812743 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-e027d390-a60e-4695-a9ab-cab3dbfa9d39 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:17:18.816734 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-e027d390-a60e-4695-a9ab-cab3dbfa9d39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:17:18.882567 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a8cd9b-1068-4814-9d82-c6e78ad91b34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:18.883487 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a8cd9b-1068-4814-9d82-c6e78ad91b34 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:18.883773 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a8cd9b-1068-4814-9d82-c6e78ad91b34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:18.884053 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a8cd9b-1068-4814-9d82-c6e78ad91b34 admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:18.884790 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a8cd9b-1068-4814-9d82-c6e78ad91b34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:18.886531 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a8cd9b-1068-4814-9d82-c6e78ad91b34 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:18.897137 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 992/1954] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:17:18 2022] GET /identity/v3/roles => generated 1420 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:17:18.903627 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a8cd9b-1068-4814-9d82-c6e78ad91b34 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:18.905468 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-93a8cd9b-1068-4814-9d82-c6e78ad91b34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:17:18.916002 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 963/1955] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:17:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:17:18.920880 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a8cd9b-1068-4814-9d82-c6e78ad91b34 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:18.923789 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5645224-18e7-4f20-a03f-8b5cf694791b None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:18.924084 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5645224-18e7-4f20-a03f-8b5cf694791b None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:18.924362 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5645224-18e7-4f20-a03f-8b5cf694791b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:18.926943 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e1f77b-3414-4008-9924-2d668ce98fc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:18.927794 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e1f77b-3414-4008-9924-2d668ce98fc7 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:18.928356 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e1f77b-3414-4008-9924-2d668ce98fc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:18.929280 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e1f77b-3414-4008-9924-2d668ce98fc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:18.930257 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e1f77b-3414-4008-9924-2d668ce98fc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:18.932241 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e1f77b-3414-4008-9924-2d668ce98fc7 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:18.943577 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 993/1956] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:17:18 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:17:18.946505 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-c5645224-18e7-4f20-a03f-8b5cf694791b None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:17:18.948026 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-c5645224-18e7-4f20-a03f-8b5cf694791b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:17:18.948343 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e1f77b-3414-4008-9924-2d668ce98fc7 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:18.957302 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 964/1957] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:17:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 18 05:17:18.964639 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5645224-18e7-4f20-a03f-8b5cf694791b None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:18.964962 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e1f77b-3414-4008-9924-2d668ce98fc7 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:18.969892 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6583c2dc-163f-48e7-9291-1d70bf349876 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:18.970675 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6583c2dc-163f-48e7-9291-1d70bf349876 admin admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:18.971008 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6583c2dc-163f-48e7-9291-1d70bf349876 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:18.971289 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6583c2dc-163f-48e7-9291-1d70bf349876 admin admin] PATH_INFO: `/v3/projects/d3a208271e6a4fca8a1de25c539460c8/users/a7cf4a2e5fa746f79790f3e05ca8fee5/roles/cbf5c4c26898413d93e73b46b41046a3` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:18.976058 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5645224-18e7-4f20-a03f-8b5cf694791b None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:18.981203 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b73a86-d17f-45aa-b6b4-bb9e08a30ea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:18.982040 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b73a86-d17f-45aa-b6b4-bb9e08a30ea5 admin admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:18.982426 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b73a86-d17f-45aa-b6b4-bb9e08a30ea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:18.982891 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b73a86-d17f-45aa-b6b4-bb9e08a30ea5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:18.983632 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b73a86-d17f-45aa-b6b4-bb9e08a30ea5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 05:17:18.984094 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b73a86-d17f-45aa-b6b4-bb9e08a30ea5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:18.986051 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b73a86-d17f-45aa-b6b4-bb9e08a30ea5 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:18.987726 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6583c2dc-163f-48e7-9291-1d70bf349876 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3a208271e6a4fca8a1de25c539460c8, user_id=a7cf4a2e5fa746f79790f3e05ca8fee5, role_id=cbf5c4c26898413d93e73b46b41046a3)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:18.992683 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6583c2dc-163f-48e7-9291-1d70bf349876 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:19.013450 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 965/1958] 10.222.0.23 () {58 vars in 1152 bytes} [Sat Jun 18 05:17:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 18 05:17:19.016553 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b73a86-d17f-45aa-b6b4-bb9e08a30ea5 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:19.026903 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b73a86-d17f-45aa-b6b4-bb9e08a30ea5 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:19.031618 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97aeb38f-d001-4c34-b413-38d9cd8d1bc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:19.032259 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aeb38f-d001-4c34-b413-38d9cd8d1bc0 admin admin] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:19.032546 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aeb38f-d001-4c34-b413-38d9cd8d1bc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:19.032827 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aeb38f-d001-4c34-b413-38d9cd8d1bc0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:19.033580 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97aeb38f-d001-4c34-b413-38d9cd8d1bc0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:19.035542 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97aeb38f-d001-4c34-b413-38d9cd8d1bc0 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:19.098792 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 994/1959] 10.222.0.23 () {60 vars in 1482 bytes} [Sat Jun 18 05:17:18 2022] PUT /identity/v3/projects/d3a208271e6a4fca8a1de25c539460c8/users/a7cf4a2e5fa746f79790f3e05ca8fee5/roles/cbf5c4c26898413d93e73b46b41046a3 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:17:19.109876 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91157b9b-2da5-4868-9f5c-d0f1cf0df97c None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:19.110171 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91157b9b-2da5-4868-9f5c-d0f1cf0df97c None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:19.110441 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91157b9b-2da5-4868-9f5c-d0f1cf0df97c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:19.120460 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:17:19.129324 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 966/1960] 10.222.0.23 () {60 vars in 1152 bytes} [Sat Jun 18 05:17:19 2022] POST /identity/v3/projects => generated 377 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 18 05:17:19.133952 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97aeb38f-d001-4c34-b413-38d9cd8d1bc0 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:19.145705 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-91157b9b-2da5-4868-9f5c-d0f1cf0df97c None None] MFA Rules not processed for user `a7cf4a2e5fa746f79790f3e05ca8fee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:17:19.151056 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-91157b9b-2da5-4868-9f5c-d0f1cf0df97c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:17:19.203508 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97aeb38f-d001-4c34-b413-38d9cd8d1bc0 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:19.210153 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5040329-6ec1-4cba-bbfd-501c5a1ce5cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:19.210901 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5040329-6ec1-4cba-bbfd-501c5a1ce5cb admin admin] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:19.211185 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5040329-6ec1-4cba-bbfd-501c5a1ce5cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:19.211464 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5040329-6ec1-4cba-bbfd-501c5a1ce5cb admin admin] PATH_INFO: `/v3/users` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:19.212174 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5040329-6ec1-4cba-bbfd-501c5a1ce5cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:19.214571 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5040329-6ec1-4cba-bbfd-501c5a1ce5cb admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:19.219343 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 995/1961] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:17:19 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 18 05:17:19.232285 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91157b9b-2da5-4868-9f5c-d0f1cf0df97c None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:19.233798 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-91157b9b-2da5-4868-9f5c-d0f1cf0df97c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:17:19.294441 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91157b9b-2da5-4868-9f5c-d0f1cf0df97c None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:19.300449 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a93d58-6d34-4d48-ba2d-5d4752354c22 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9de0428abc8400aa338aba4f7c34c94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:19.301137 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a93d58-6d34-4d48-ba2d-5d4752354c22 service cinder] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:19.301444 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a93d58-6d34-4d48-ba2d-5d4752354c22 service cinder] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:19.301723 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a93d58-6d34-4d48-ba2d-5d4752354c22 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:19.311804 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a93d58-6d34-4d48-ba2d-5d4752354c22 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:19.313331 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a93d58-6d34-4d48-ba2d-5d4752354c22 service cinder] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:19.324433 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 996/1962] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:17:19 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 18 05:17:19.349444 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a93d58-6d34-4d48-ba2d-5d4752354c22 service cinder] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:19.362402 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a93d58-6d34-4d48-ba2d-5d4752354c22 service cinder] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:19.368402 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83b1167-2f1f-4705-a1ff-514698a71dcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:19.369476 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83b1167-2f1f-4705-a1ff-514698a71dcc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:19.372810 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83b1167-2f1f-4705-a1ff-514698a71dcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:19.372968 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 967/1963] 10.222.0.23 () {60 vars in 1143 bytes} [Sat Jun 18 05:17:19 2022] POST /identity/v3/users => generated 289 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 18 05:17:19.373065 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83b1167-2f1f-4705-a1ff-514698a71dcc admin admin] PATH_INFO: `/v3/users/a7cf4a2e5fa746f79790f3e05ca8fee5` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:19.373065 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83b1167-2f1f-4705-a1ff-514698a71dcc admin admin] RBAC: Authorizing `identity:delete_user(user_id=a7cf4a2e5fa746f79790f3e05ca8fee5)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:19.374223 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83b1167-2f1f-4705-a1ff-514698a71dcc admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:19.384911 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5040329-6ec1-4cba-bbfd-501c5a1ce5cb admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:19.402657 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5040329-6ec1-4cba-bbfd-501c5a1ce5cb admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:19.408783 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a43dade-a49c-4fb0-a9d7-cf3dc43e1447 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:19.409512 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a43dade-a49c-4fb0-a9d7-cf3dc43e1447 admin admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:19.415048 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a43dade-a49c-4fb0-a9d7-cf3dc43e1447 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:19.415048 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a43dade-a49c-4fb0-a9d7-cf3dc43e1447 admin admin] PATH_INFO: `/v3/roles` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:19.415048 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a43dade-a49c-4fb0-a9d7-cf3dc43e1447 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:19.415048 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a43dade-a49c-4fb0-a9d7-cf3dc43e1447 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:19.423049 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 968/1964] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:17:19 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:17:19.426767 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a43dade-a49c-4fb0-a9d7-cf3dc43e1447 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:19.436836 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a43dade-a49c-4fb0-a9d7-cf3dc43e1447 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:19.441526 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253371e8-89b2-4e71-8cb0-d45b275cc185 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:19.442162 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253371e8-89b2-4e71-8cb0-d45b275cc185 admin admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:19.442639 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253371e8-89b2-4e71-8cb0-d45b275cc185 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:19.442748 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253371e8-89b2-4e71-8cb0-d45b275cc185 admin admin] PATH_INFO: `/v3/roles` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:19.443469 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253371e8-89b2-4e71-8cb0-d45b275cc185 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:19.445103 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253371e8-89b2-4e71-8cb0-d45b275cc185 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:19.455461 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 969/1965] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:17:19 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 18 05:17:19.459442 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253371e8-89b2-4e71-8cb0-d45b275cc185 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:19.469485 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253371e8-89b2-4e71-8cb0-d45b275cc185 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:19.476486 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f092a0e0-1946-447f-897c-1fe9c489a460 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:19.477406 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f092a0e0-1946-447f-897c-1fe9c489a460 admin admin] REQUEST_METHOD: `PUT` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:19.477685 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f092a0e0-1946-447f-897c-1fe9c489a460 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:19.477966 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f092a0e0-1946-447f-897c-1fe9c489a460 admin admin] PATH_INFO: `/v3/projects/bbe4dd7c2cd64627b0e0cee06a06275f/users/9cda29438b944702b2dad445fa604697/roles/cbf5c4c26898413d93e73b46b41046a3` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:19.494558 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f092a0e0-1946-447f-897c-1fe9c489a460 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bbe4dd7c2cd64627b0e0cee06a06275f, user_id=9cda29438b944702b2dad445fa604697, role_id=cbf5c4c26898413d93e73b46b41046a3)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:19.497272 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f092a0e0-1946-447f-897c-1fe9c489a460 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:19.624917 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 970/1966] 10.222.0.23 () {60 vars in 1482 bytes} [Sat Jun 18 05:17:19 2022] PUT /identity/v3/projects/bbe4dd7c2cd64627b0e0cee06a06275f/users/9cda29438b944702b2dad445fa604697/roles/cbf5c4c26898413d93e73b46b41046a3 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 18 05:17:19.636062 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04adee5e-26ef-4d14-b01c-6831a88f3b90 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:19.636380 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04adee5e-26ef-4d14-b01c-6831a88f3b90 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:19.636658 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04adee5e-26ef-4d14-b01c-6831a88f3b90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:19.646405 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-a83b1167-2f1f-4705-a1ff-514698a71dcc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a7cf4a2e5fa746f79790f3e05ca8fee5'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:19.676610 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-a83b1167-2f1f-4705-a1ff-514698a71dcc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a7cf4a2e5fa746f79790f3e05ca8fee5'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:19.677600 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-a83b1167-2f1f-4705-a1ff-514698a71dcc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a7cf4a2e5fa746f79790f3e05ca8fee5'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:19.700166 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-04adee5e-26ef-4d14-b01c-6831a88f3b90 None None] MFA Rules not processed for user `9cda29438b944702b2dad445fa604697`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:17:19.700166 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-04adee5e-26ef-4d14-b01c-6831a88f3b90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:17:19.751161 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-a83b1167-2f1f-4705-a1ff-514698a71dcc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a7cf4a2e5fa746f79790f3e05ca8fee5'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:19.761125 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-a83b1167-2f1f-4705-a1ff-514698a71dcc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a7cf4a2e5fa746f79790f3e05ca8fee5'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:19.768515 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 997/1967] 10.222.0.23 () {58 vars in 1223 bytes} [Sat Jun 18 05:17:19 2022] DELETE /identity/v3/users/a7cf4a2e5fa746f79790f3e05ca8fee5 => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 18 05:17:19.774417 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9ca6ac-17cd-43b8-9870-4efaf6564d32 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:19.774666 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9ca6ac-17cd-43b8-9870-4efaf6564d32 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:19.775053 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9ca6ac-17cd-43b8-9870-4efaf6564d32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:19.800478 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-af9ca6ac-17cd-43b8-9870-4efaf6564d32 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:17:19.803074 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 971/1968] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:17:19 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 18 05:17:19.803204 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-af9ca6ac-17cd-43b8-9870-4efaf6564d32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:17:19.818270 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04adee5e-26ef-4d14-b01c-6831a88f3b90 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:19.820465 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-04adee5e-26ef-4d14-b01c-6831a88f3b90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:17:19.923252 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:17:19.930109 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 998/1969] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:17:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:17:19.937414 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9ca6ac-17cd-43b8-9870-4efaf6564d32 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:19.939472 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-af9ca6ac-17cd-43b8-9870-4efaf6564d32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:17:19.972997 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04adee5e-26ef-4d14-b01c-6831a88f3b90 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:19.981787 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b928962a-609f-489b-8b5c-edf574f89518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:19.982661 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b928962a-609f-489b-8b5c-edf574f89518 admin admin] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:19.983100 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b928962a-609f-489b-8b5c-edf574f89518 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:19.983526 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b928962a-609f-489b-8b5c-edf574f89518 admin admin] PATH_INFO: `/v3/projects` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:19.984568 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b928962a-609f-489b-8b5c-edf574f89518 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:19.986742 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b928962a-609f-489b-8b5c-edf574f89518 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:20.068852 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9ca6ac-17cd-43b8-9870-4efaf6564d32 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:20.079344 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26f46fd-f34c-47a3-87f9-1c552a5ff4a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c4d7cde8f4ad9aa479bb09e325db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:20.080650 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26f46fd-f34c-47a3-87f9-1c552a5ff4a9 service neutron] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:20.081343 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26f46fd-f34c-47a3-87f9-1c552a5ff4a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:20.081937 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26f46fd-f34c-47a3-87f9-1c552a5ff4a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:20.091257 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 972/1970] 10.222.0.23 () {60 vars in 1152 bytes} [Sat Jun 18 05:17:19 2022] POST /identity/v3/projects => generated 377 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 18 05:17:20.091257 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b928962a-609f-489b-8b5c-edf574f89518 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:20.095500 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26f46fd-f34c-47a3-87f9-1c552a5ff4a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:20.097872 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26f46fd-f34c-47a3-87f9-1c552a5ff4a9 service neutron] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:20.140722 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 999/1971] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:17:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 18 05:17:20.255138 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b928962a-609f-489b-8b5c-edf574f89518 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:20.262868 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56360775-f9a5-4ddd-9a8b-3f959b420dff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:20.267336 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56360775-f9a5-4ddd-9a8b-3f959b420dff admin admin] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:20.267336 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56360775-f9a5-4ddd-9a8b-3f959b420dff admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:20.267336 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56360775-f9a5-4ddd-9a8b-3f959b420dff admin admin] PATH_INFO: `/v3/users` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:20.267336 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56360775-f9a5-4ddd-9a8b-3f959b420dff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:20.267336 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56360775-f9a5-4ddd-9a8b-3f959b420dff admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:20.354254 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 973/1972] 10.222.0.23 () {60 vars in 1143 bytes} [Sat Jun 18 05:17:20 2022] POST /identity/v3/users => generated 295 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 18 05:17:20.355702 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26f46fd-f34c-47a3-87f9-1c552a5ff4a9 service neutron] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:20.365846 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26f46fd-f34c-47a3-87f9-1c552a5ff4a9 service neutron] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:20.370919 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a66637-7638-410d-a274-d9a6ac9078fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:20.371587 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a66637-7638-410d-a274-d9a6ac9078fb admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:20.371870 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a66637-7638-410d-a274-d9a6ac9078fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:20.372143 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a66637-7638-410d-a274-d9a6ac9078fb admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:20.379208 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a66637-7638-410d-a274-d9a6ac9078fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:20.381034 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a66637-7638-410d-a274-d9a6ac9078fb admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:20.399812 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1000/1973] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:17:20 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:17:20.402449 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56360775-f9a5-4ddd-9a8b-3f959b420dff admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:20.423468 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56360775-f9a5-4ddd-9a8b-3f959b420dff admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:20.432437 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d16025c-eaaa-4619-882d-e98f2e0fb8da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:20.435904 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d16025c-eaaa-4619-882d-e98f2e0fb8da admin admin] REQUEST_METHOD: `PUT` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:20.436561 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d16025c-eaaa-4619-882d-e98f2e0fb8da admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:20.437688 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d16025c-eaaa-4619-882d-e98f2e0fb8da admin admin] PATH_INFO: `/v3/projects/dabc40ef707d4a5fad758723cd168c5a/users/08e5cdcfa3074129824b59621f440bb5/roles/21c65f2c51bd4dd283d13507d2714467` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:20.492103 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:17:20.508918 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d16025c-eaaa-4619-882d-e98f2e0fb8da admin admin] RBAC: Authorizing `identity:create_grant(project_id=dabc40ef707d4a5fad758723cd168c5a, user_id=08e5cdcfa3074129824b59621f440bb5, role_id=21c65f2c51bd4dd283d13507d2714467)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:20.531475 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d16025c-eaaa-4619-882d-e98f2e0fb8da admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:20.610116 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 974/1974] 10.222.0.23 () {60 vars in 1482 bytes} [Sat Jun 18 05:17:20 2022] PUT /identity/v3/projects/dabc40ef707d4a5fad758723cd168c5a/users/08e5cdcfa3074129824b59621f440bb5/roles/21c65f2c51bd4dd283d13507d2714467 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 18 05:17:20.614882 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a66637-7638-410d-a274-d9a6ac9078fb admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:20.626646 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a66637-7638-410d-a274-d9a6ac9078fb admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:20.715851 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be88346-3236-4225-9c6b-00db339b1501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:20.716563 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be88346-3236-4225-9c6b-00db339b1501 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:20.716849 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be88346-3236-4225-9c6b-00db339b1501 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:20.717739 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be88346-3236-4225-9c6b-00db339b1501 admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:20.717888 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be88346-3236-4225-9c6b-00db339b1501 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:20.719511 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be88346-3236-4225-9c6b-00db339b1501 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:20.733129 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1001/1975] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:17:20 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:17:20.736852 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d16025c-eaaa-4619-882d-e98f2e0fb8da admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:20.754451 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d16025c-eaaa-4619-882d-e98f2e0fb8da admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:20.760910 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49014a78-d7e2-4e97-9593-4b3273869b42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:20.763054 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49014a78-d7e2-4e97-9593-4b3273869b42 admin admin] REQUEST_METHOD: `PUT` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:20.763054 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49014a78-d7e2-4e97-9593-4b3273869b42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:20.763054 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49014a78-d7e2-4e97-9593-4b3273869b42 admin admin] PATH_INFO: `/v3/projects/dabc40ef707d4a5fad758723cd168c5a/users/08e5cdcfa3074129824b59621f440bb5/roles/21c65f2c51bd4dd283d13507d2714467` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:20.764785 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49014a78-d7e2-4e97-9593-4b3273869b42 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dabc40ef707d4a5fad758723cd168c5a, user_id=08e5cdcfa3074129824b59621f440bb5, role_id=21c65f2c51bd4dd283d13507d2714467)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:20.767742 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49014a78-d7e2-4e97-9593-4b3273869b42 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:20.781896 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 975/1976] 10.222.0.23 () {60 vars in 1482 bytes} [Sat Jun 18 05:17:20 2022] PUT /identity/v3/projects/dabc40ef707d4a5fad758723cd168c5a/users/08e5cdcfa3074129824b59621f440bb5/roles/21c65f2c51bd4dd283d13507d2714467 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:17:20.793571 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b82339-9c23-4462-b45b-16a299f9f0b9 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:20.793926 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b82339-9c23-4462-b45b-16a299f9f0b9 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:20.794282 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b82339-9c23-4462-b45b-16a299f9f0b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:20.829017 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:17:20.830470 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49014a78-d7e2-4e97-9593-4b3273869b42 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:20.832330 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-49014a78-d7e2-4e97-9593-4b3273869b42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:17:20.863344 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-22b82339-9c23-4462-b45b-16a299f9f0b9 None None] MFA Rules not processed for user `08e5cdcfa3074129824b59621f440bb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:17:20.868664 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-22b82339-9c23-4462-b45b-16a299f9f0b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:17:20.972098 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49014a78-d7e2-4e97-9593-4b3273869b42 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:20.990319 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7376b5e7-b84e-4d3b-9444-ed5a495b6d02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:20.994859 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7376b5e7-b84e-4d3b-9444-ed5a495b6d02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:20.994859 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7376b5e7-b84e-4d3b-9444-ed5a495b6d02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:20.994859 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7376b5e7-b84e-4d3b-9444-ed5a495b6d02 admin admin] PATH_INFO: `/v3/projects/d3a208271e6a4fca8a1de25c539460c8` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:20.994859 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7376b5e7-b84e-4d3b-9444-ed5a495b6d02 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d3a208271e6a4fca8a1de25c539460c8)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:20.997394 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7376b5e7-b84e-4d3b-9444-ed5a495b6d02 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:21.022360 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1002/1977] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:17:20 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:17:21.034397 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b82339-9c23-4462-b45b-16a299f9f0b9 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:21.036590 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-22b82339-9c23-4462-b45b-16a299f9f0b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:17:21.129315 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b82339-9c23-4462-b45b-16a299f9f0b9 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:21.158136 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-7376b5e7-b84e-4d3b-9444-ed5a495b6d02 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd3a208271e6a4fca8a1de25c539460c8', 'request_id': 'req-7376b5e7-b84e-4d3b-9444-ed5a495b6d02'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:21.171346 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-7376b5e7-b84e-4d3b-9444-ed5a495b6d02 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd3a208271e6a4fca8a1de25c539460c8', 'request_id': 'req-7376b5e7-b84e-4d3b-9444-ed5a495b6d02'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:21.204044 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6ad9e6-11c7-4467-9d8d-b6f6e6cd06e8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9de0428abc8400aa338aba4f7c34c94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:21.204908 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6ad9e6-11c7-4467-9d8d-b6f6e6cd06e8 service cinder] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:21.208579 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6ad9e6-11c7-4467-9d8d-b6f6e6cd06e8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:21.208579 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6ad9e6-11c7-4467-9d8d-b6f6e6cd06e8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:21.259352 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-7376b5e7-b84e-4d3b-9444-ed5a495b6d02 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd3a208271e6a4fca8a1de25c539460c8', 'request_id': 'req-7376b5e7-b84e-4d3b-9444-ed5a495b6d02'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:21.280909 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6ad9e6-11c7-4467-9d8d-b6f6e6cd06e8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:21.282692 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6ad9e6-11c7-4467-9d8d-b6f6e6cd06e8 service cinder] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:21.296515 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1003/1978] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:17:21 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 18 05:17:21.306333 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-7376b5e7-b84e-4d3b-9444-ed5a495b6d02 admin admin] The token cache is being invalidate because project d3a208271e6a4fca8a1de25c539460c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80468) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 18 05:17:21.311080 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-7376b5e7-b84e-4d3b-9444-ed5a495b6d02 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:21.312156 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-7376b5e7-b84e-4d3b-9444-ed5a495b6d02 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:21.315120 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 976/1979] 10.222.0.23 () {58 vars in 1232 bytes} [Sat Jun 18 05:17:20 2022] DELETE /identity/v3/projects/d3a208271e6a4fca8a1de25c539460c8 => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 18 05:17:21.319986 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6271ca52-6f47-47cd-bca3-a29d7040de5b None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:21.320699 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6271ca52-6f47-47cd-bca3-a29d7040de5b None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:21.320992 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6271ca52-6f47-47cd-bca3-a29d7040de5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:21.373849 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-6271ca52-6f47-47cd-bca3-a29d7040de5b None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:17:21.378403 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-6271ca52-6f47-47cd-bca3-a29d7040de5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:17:21.505085 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1004/1980] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:17:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:17:21.508107 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5032072d-4a79-42e4-9b75-44b828ca27e6 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:21.512212 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5032072d-4a79-42e4-9b75-44b828ca27e6 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:21.512517 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5032072d-4a79-42e4-9b75-44b828ca27e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:21.534679 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-5032072d-4a79-42e4-9b75-44b828ca27e6 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:17:21.535911 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-5032072d-4a79-42e4-9b75-44b828ca27e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:17:21.553215 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 977/1981] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:17:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:17:21.555151 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6271ca52-6f47-47cd-bca3-a29d7040de5b None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:21.565752 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6271ca52-6f47-47cd-bca3-a29d7040de5b None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:21.573823 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21da5c90-86e3-4625-b724-77e26f217b1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:21.578685 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21da5c90-86e3-4625-b724-77e26f217b1f admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:21.578685 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21da5c90-86e3-4625-b724-77e26f217b1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:21.578685 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21da5c90-86e3-4625-b724-77e26f217b1f admin admin] PATH_INFO: `/v3/domains` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:21.578685 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21da5c90-86e3-4625-b724-77e26f217b1f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 05:17:21.578685 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21da5c90-86e3-4625-b724-77e26f217b1f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:21.578685 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21da5c90-86e3-4625-b724-77e26f217b1f admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:21.595548 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1005/1982] 10.222.0.23 () {58 vars in 1152 bytes} [Sat Jun 18 05:17:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 05:17:21.611250 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5032072d-4a79-42e4-9b75-44b828ca27e6 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:21.621271 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5032072d-4a79-42e4-9b75-44b828ca27e6 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:21.626415 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3168a1-003d-44da-906a-20249610b79a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:21.627098 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3168a1-003d-44da-906a-20249610b79a admin admin] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:21.627374 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3168a1-003d-44da-906a-20249610b79a admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:21.627654 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3168a1-003d-44da-906a-20249610b79a admin admin] PATH_INFO: `/v3/projects` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:21.628548 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3168a1-003d-44da-906a-20249610b79a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:21.630650 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3168a1-003d-44da-906a-20249610b79a admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:21.725091 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 978/1983] 10.222.0.23 () {60 vars in 1152 bytes} [Sat Jun 18 05:17:21 2022] POST /identity/v3/projects => generated 375 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 18 05:17:21.729581 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21da5c90-86e3-4625-b724-77e26f217b1f admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:21.769194 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21da5c90-86e3-4625-b724-77e26f217b1f admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:21.802900 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:INSERT=6,keystone:SELECT=86,keystone:DELETE=8 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:17:21.873408 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ee9bb7-d17e-4988-a2d3-e50e8b6507aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:21.875191 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ee9bb7-d17e-4988-a2d3-e50e8b6507aa admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:21.875191 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ee9bb7-d17e-4988-a2d3-e50e8b6507aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:21.875481 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ee9bb7-d17e-4988-a2d3-e50e8b6507aa admin admin] PATH_INFO: `/v3/users` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:21.877424 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ee9bb7-d17e-4988-a2d3-e50e8b6507aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:21.889321 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ee9bb7-d17e-4988-a2d3-e50e8b6507aa admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:21.957513 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1006/1984] 10.222.0.23 () {60 vars in 1143 bytes} [Sat Jun 18 05:17:21 2022] POST /identity/v3/users => generated 288 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 18 05:17:21.962386 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3168a1-003d-44da-906a-20249610b79a admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:21.973165 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3168a1-003d-44da-906a-20249610b79a admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:21.978294 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067bbfcc-bb34-4cae-8847-f210e8e9a82a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:21.979158 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067bbfcc-bb34-4cae-8847-f210e8e9a82a admin admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:21.979444 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067bbfcc-bb34-4cae-8847-f210e8e9a82a admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:21.979717 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067bbfcc-bb34-4cae-8847-f210e8e9a82a admin admin] PATH_INFO: `/v3/roles` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:21.980455 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067bbfcc-bb34-4cae-8847-f210e8e9a82a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:21.982083 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067bbfcc-bb34-4cae-8847-f210e8e9a82a admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:21.993750 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 979/1985] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:17:21 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:17:21.998044 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ee9bb7-d17e-4988-a2d3-e50e8b6507aa admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:22.009002 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ee9bb7-d17e-4988-a2d3-e50e8b6507aa admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:22.015293 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52c2c0e-63a6-4181-a759-406960b24e6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:22.015968 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52c2c0e-63a6-4181-a759-406960b24e6f admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:22.016248 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52c2c0e-63a6-4181-a759-406960b24e6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:22.016522 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52c2c0e-63a6-4181-a759-406960b24e6f admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:22.017252 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52c2c0e-63a6-4181-a759-406960b24e6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:22.019015 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52c2c0e-63a6-4181-a759-406960b24e6f admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:22.031497 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1007/1986] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:17:21 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:17:22.035965 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067bbfcc-bb34-4cae-8847-f210e8e9a82a admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:22.047190 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067bbfcc-bb34-4cae-8847-f210e8e9a82a admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:22.052479 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e360ac6-8def-429a-be90-6802a6753396 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:22.053656 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e360ac6-8def-429a-be90-6802a6753396 admin admin] REQUEST_METHOD: `PUT` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:22.054280 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e360ac6-8def-429a-be90-6802a6753396 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:22.054583 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e360ac6-8def-429a-be90-6802a6753396 admin admin] PATH_INFO: `/v3/projects/8fa0c1bb956f4efbac54fdfb60f6573c/users/b17031a1b654414a9737632fa471c1b8/roles/cbf5c4c26898413d93e73b46b41046a3` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:22.077339 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e360ac6-8def-429a-be90-6802a6753396 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8fa0c1bb956f4efbac54fdfb60f6573c, user_id=b17031a1b654414a9737632fa471c1b8, role_id=cbf5c4c26898413d93e73b46b41046a3)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:22.080554 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e360ac6-8def-429a-be90-6802a6753396 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:22.170051 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 980/1987] 10.222.0.23 () {60 vars in 1482 bytes} [Sat Jun 18 05:17:22 2022] PUT /identity/v3/projects/8fa0c1bb956f4efbac54fdfb60f6573c/users/b17031a1b654414a9737632fa471c1b8/roles/cbf5c4c26898413d93e73b46b41046a3 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:17:22.183079 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e11c508-9af0-4a4e-8243-2d2c34eb83a6 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:22.184073 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e11c508-9af0-4a4e-8243-2d2c34eb83a6 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:22.184492 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e11c508-9af0-4a4e-8243-2d2c34eb83a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:22.216909 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-4e11c508-9af0-4a4e-8243-2d2c34eb83a6 None None] MFA Rules not processed for user `b17031a1b654414a9737632fa471c1b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:17:22.218680 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-4e11c508-9af0-4a4e-8243-2d2c34eb83a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:17:22.296737 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1008/1988] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:17:22 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:17:22.413691 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:17:22.518214 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e360ac6-8def-429a-be90-6802a6753396 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:22.520077 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-0e360ac6-8def-429a-be90-6802a6753396 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:17:22.579546 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e360ac6-8def-429a-be90-6802a6753396 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:22.583161 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb04c8db-fb5d-4b42-8681-e18dddc6d75f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9de0428abc8400aa338aba4f7c34c94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:22.584049 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb04c8db-fb5d-4b42-8681-e18dddc6d75f service cinder] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:22.584341 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb04c8db-fb5d-4b42-8681-e18dddc6d75f service cinder] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:22.585489 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb04c8db-fb5d-4b42-8681-e18dddc6d75f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:22.594154 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb04c8db-fb5d-4b42-8681-e18dddc6d75f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:22.595710 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb04c8db-fb5d-4b42-8681-e18dddc6d75f service cinder] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:22.605659 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 981/1989] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:17:22 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:17:23.146047 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:17:23.386803 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:17:23.433295 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e11c508-9af0-4a4e-8243-2d2c34eb83a6 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:23.435176 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-4e11c508-9af0-4a4e-8243-2d2c34eb83a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:17:23.523858 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e11c508-9af0-4a4e-8243-2d2c34eb83a6 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:23.534350 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3f90e6-dfbe-4f5e-90a7-2f44207a808c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:23.535935 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3f90e6-dfbe-4f5e-90a7-2f44207a808c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:23.536428 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3f90e6-dfbe-4f5e-90a7-2f44207a808c admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:23.537010 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3f90e6-dfbe-4f5e-90a7-2f44207a808c admin admin] PATH_INFO: `/v3/users/c55c55e90f2b4a73bbdf3e9ffaf9a75f` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:23.539734 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3f90e6-dfbe-4f5e-90a7-2f44207a808c admin admin] RBAC: Authorizing `identity:delete_user(user_id=c55c55e90f2b4a73bbdf3e9ffaf9a75f)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:23.542033 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3f90e6-dfbe-4f5e-90a7-2f44207a808c admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:23.745542 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-1a3f90e6-dfbe-4f5e-90a7-2f44207a808c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c55c55e90f2b4a73bbdf3e9ffaf9a75f'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:23.762966 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-1a3f90e6-dfbe-4f5e-90a7-2f44207a808c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c55c55e90f2b4a73bbdf3e9ffaf9a75f'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:23.764023 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-1a3f90e6-dfbe-4f5e-90a7-2f44207a808c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c55c55e90f2b4a73bbdf3e9ffaf9a75f'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:23.825880 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-1a3f90e6-dfbe-4f5e-90a7-2f44207a808c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c55c55e90f2b4a73bbdf3e9ffaf9a75f'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:23.835523 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-1a3f90e6-dfbe-4f5e-90a7-2f44207a808c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c55c55e90f2b4a73bbdf3e9ffaf9a75f'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:23.848923 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1009/1990] 10.222.0.23 () {58 vars in 1223 bytes} [Sat Jun 18 05:17:23 2022] DELETE /identity/v3/users/c55c55e90f2b4a73bbdf3e9ffaf9a75f => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:17:23.853653 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdce9d4-bc38-44ab-9364-613dc8acdd65 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:23.854149 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdce9d4-bc38-44ab-9364-613dc8acdd65 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:23.854257 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdce9d4-bc38-44ab-9364-613dc8acdd65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:23.877740 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-5cdce9d4-bc38-44ab-9364-613dc8acdd65 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:17:23.879916 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-5cdce9d4-bc38-44ab-9364-613dc8acdd65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:17:23.979700 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 982/1991] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:17:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:17:23.989910 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3f90e6-dfbe-4f5e-90a7-2f44207a808c admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:23.991707 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-1a3f90e6-dfbe-4f5e-90a7-2f44207a808c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:17:24.073875 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3f90e6-dfbe-4f5e-90a7-2f44207a808c admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:24.078779 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fd84e3-f0ab-4e04-8c80-1cfb2e4d5cf5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c4d7cde8f4ad9aa479bb09e325db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:24.079643 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fd84e3-f0ab-4e04-8c80-1cfb2e4d5cf5 service neutron] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:24.080152 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fd84e3-f0ab-4e04-8c80-1cfb2e4d5cf5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:24.080599 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fd84e3-f0ab-4e04-8c80-1cfb2e4d5cf5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:24.093516 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fd84e3-f0ab-4e04-8c80-1cfb2e4d5cf5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:24.095248 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fd84e3-f0ab-4e04-8c80-1cfb2e4d5cf5 service neutron] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:24.107978 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1010/1992] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:17:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:17:24.346652 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:17:24.568028 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:17:24.620524 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdce9d4-bc38-44ab-9364-613dc8acdd65 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:24.622141 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-5cdce9d4-bc38-44ab-9364-613dc8acdd65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:17:24.636147 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdce9d4-bc38-44ab-9364-613dc8acdd65 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:24.641275 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d803ed3-de1a-402a-8366-84707649792b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:24.642201 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d803ed3-de1a-402a-8366-84707649792b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:24.642488 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d803ed3-de1a-402a-8366-84707649792b admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:24.642846 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d803ed3-de1a-402a-8366-84707649792b admin admin] PATH_INFO: `/v3/projects/bc024382b15c40e2bbb7cf58f1906fb9` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:24.644096 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d803ed3-de1a-402a-8366-84707649792b admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc024382b15c40e2bbb7cf58f1906fb9)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:24.646141 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d803ed3-de1a-402a-8366-84707649792b admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:24.828063 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-9d803ed3-de1a-402a-8366-84707649792b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc024382b15c40e2bbb7cf58f1906fb9', 'request_id': 'req-9d803ed3-de1a-402a-8366-84707649792b'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:24.833534 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-9d803ed3-de1a-402a-8366-84707649792b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc024382b15c40e2bbb7cf58f1906fb9', 'request_id': 'req-9d803ed3-de1a-402a-8366-84707649792b'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:24.927696 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-9d803ed3-de1a-402a-8366-84707649792b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc024382b15c40e2bbb7cf58f1906fb9', 'request_id': 'req-9d803ed3-de1a-402a-8366-84707649792b'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:24.940514 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-9d803ed3-de1a-402a-8366-84707649792b admin admin] The token cache is being invalidate because project bc024382b15c40e2bbb7cf58f1906fb9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80468) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 18 05:17:24.940798 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-9d803ed3-de1a-402a-8366-84707649792b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:24.941447 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-9d803ed3-de1a-402a-8366-84707649792b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:24.942985 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 983/1993] 10.222.0.23 () {58 vars in 1232 bytes} [Sat Jun 18 05:17:24 2022] DELETE /identity/v3/projects/bc024382b15c40e2bbb7cf58f1906fb9 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:17:24.947314 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fd84e3-f0ab-4e04-8c80-1cfb2e4d5cf5 service neutron] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:24.959294 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fd84e3-f0ab-4e04-8c80-1cfb2e4d5cf5 service neutron] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:25.016801 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac2aff5-3033-4eeb-87ca-1b558ea2ace7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:25.017897 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac2aff5-3033-4eeb-87ca-1b558ea2ace7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:25.018010 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac2aff5-3033-4eeb-87ca-1b558ea2ace7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:25.018313 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac2aff5-3033-4eeb-87ca-1b558ea2ace7 admin admin] PATH_INFO: `/v3/users/705f8a19f6254cff84510d9425aa3ce4` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:25.019511 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac2aff5-3033-4eeb-87ca-1b558ea2ace7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=705f8a19f6254cff84510d9425aa3ce4)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:25.021526 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac2aff5-3033-4eeb-87ca-1b558ea2ace7 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:25.198609 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-7ac2aff5-3033-4eeb-87ca-1b558ea2ace7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '705f8a19f6254cff84510d9425aa3ce4'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:25.210630 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-7ac2aff5-3033-4eeb-87ca-1b558ea2ace7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '705f8a19f6254cff84510d9425aa3ce4'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:25.211283 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-7ac2aff5-3033-4eeb-87ca-1b558ea2ace7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '705f8a19f6254cff84510d9425aa3ce4'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:25.213544 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:17:25.322361 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-7ac2aff5-3033-4eeb-87ca-1b558ea2ace7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '705f8a19f6254cff84510d9425aa3ce4'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:25.329986 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-7ac2aff5-3033-4eeb-87ca-1b558ea2ace7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '705f8a19f6254cff84510d9425aa3ce4'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:25.337143 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1011/1994] 10.222.0.23 () {58 vars in 1223 bytes} [Sat Jun 18 05:17:24 2022] DELETE /identity/v3/users/705f8a19f6254cff84510d9425aa3ce4 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:17:25.380090 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:17:25.574171 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:17:25.631937 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=83,keystone:DELETE=11,keystone:INSERT=6 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:17:25.718143 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d803ed3-de1a-402a-8366-84707649792b admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:25.719566 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-9d803ed3-de1a-402a-8366-84707649792b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:17:25.734244 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d803ed3-de1a-402a-8366-84707649792b admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:25.793962 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27be3aac-cc95-4ad8-8e04-1c7694a74367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:25.794861 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27be3aac-cc95-4ad8-8e04-1c7694a74367 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:25.795144 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27be3aac-cc95-4ad8-8e04-1c7694a74367 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:25.795416 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27be3aac-cc95-4ad8-8e04-1c7694a74367 admin admin] PATH_INFO: `/v3/projects/018458a4a37442dc975b845b4213e9b7` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:25.796590 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27be3aac-cc95-4ad8-8e04-1c7694a74367 admin admin] RBAC: Authorizing `identity:delete_project(project_id=018458a4a37442dc975b845b4213e9b7)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:25.798530 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27be3aac-cc95-4ad8-8e04-1c7694a74367 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:25.889557 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-27be3aac-cc95-4ad8-8e04-1c7694a74367 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '018458a4a37442dc975b845b4213e9b7', 'request_id': 'req-27be3aac-cc95-4ad8-8e04-1c7694a74367'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:25.895179 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-27be3aac-cc95-4ad8-8e04-1c7694a74367 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '018458a4a37442dc975b845b4213e9b7', 'request_id': 'req-27be3aac-cc95-4ad8-8e04-1c7694a74367'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:25.922401 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:17:25.984319 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-27be3aac-cc95-4ad8-8e04-1c7694a74367 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '018458a4a37442dc975b845b4213e9b7', 'request_id': 'req-27be3aac-cc95-4ad8-8e04-1c7694a74367'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:25.996540 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-27be3aac-cc95-4ad8-8e04-1c7694a74367 admin admin] The token cache is being invalidate because project 018458a4a37442dc975b845b4213e9b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80468) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 18 05:17:25.996829 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-27be3aac-cc95-4ad8-8e04-1c7694a74367 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:25.997497 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-27be3aac-cc95-4ad8-8e04-1c7694a74367 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:25.999070 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 984/1995] 10.222.0.23 () {58 vars in 1232 bytes} [Sat Jun 18 05:17:25 2022] DELETE /identity/v3/projects/018458a4a37442dc975b845b4213e9b7 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:17:26.003923 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0dcb40-271a-4a27-b5bf-7816a328bbfd None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:26.004196 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0dcb40-271a-4a27-b5bf-7816a328bbfd None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:26.004470 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0dcb40-271a-4a27-b5bf-7816a328bbfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:26.025257 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-0d0dcb40-271a-4a27-b5bf-7816a328bbfd None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:17:26.026495 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-0d0dcb40-271a-4a27-b5bf-7816a328bbfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:17:26.085787 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1012/1996] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:17:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:17:26.093585 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bac03ec-6ec2-49a6-b33e-80edb499cca4 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:26.093881 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bac03ec-6ec2-49a6-b33e-80edb499cca4 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:26.094178 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bac03ec-6ec2-49a6-b33e-80edb499cca4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:26.112585 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-7bac03ec-6ec2-49a6-b33e-80edb499cca4 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:17:26.113832 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-7bac03ec-6ec2-49a6-b33e-80edb499cca4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:17:26.122839 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 985/1997] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:17:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:17:26.126866 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0dcb40-271a-4a27-b5bf-7816a328bbfd None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:26.136919 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0dcb40-271a-4a27-b5bf-7816a328bbfd None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:26.141639 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9285579e-44ed-435f-a2ea-0713f919b60c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:26.142324 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9285579e-44ed-435f-a2ea-0713f919b60c admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:26.142630 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9285579e-44ed-435f-a2ea-0713f919b60c admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:26.142904 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9285579e-44ed-435f-a2ea-0713f919b60c admin admin] PATH_INFO: `/v3/domains` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:26.143681 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9285579e-44ed-435f-a2ea-0713f919b60c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 05:17:26.144011 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9285579e-44ed-435f-a2ea-0713f919b60c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:26.145636 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9285579e-44ed-435f-a2ea-0713f919b60c admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:26.157394 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1013/1998] 10.222.0.23 () {58 vars in 1152 bytes} [Sat Jun 18 05:17:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 05:17:26.161697 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bac03ec-6ec2-49a6-b33e-80edb499cca4 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:26.161872 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:17:26.170978 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bac03ec-6ec2-49a6-b33e-80edb499cca4 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:26.175667 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67425f19-fcce-4c19-96ec-e889786f5aef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:26.176310 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67425f19-fcce-4c19-96ec-e889786f5aef admin admin] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:26.176601 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67425f19-fcce-4c19-96ec-e889786f5aef admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:26.176877 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67425f19-fcce-4c19-96ec-e889786f5aef admin admin] PATH_INFO: `/v3/projects` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:26.177652 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67425f19-fcce-4c19-96ec-e889786f5aef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:26.180152 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67425f19-fcce-4c19-96ec-e889786f5aef admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:26.283678 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 986/1999] 10.222.0.23 () {60 vars in 1152 bytes} [Sat Jun 18 05:17:26 2022] POST /identity/v3/projects => generated 387 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:17:26.288486 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9285579e-44ed-435f-a2ea-0713f919b60c admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:26.298902 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9285579e-44ed-435f-a2ea-0713f919b60c admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:26.339701 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:17:26.354943 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fc30b8-7e82-4418-bff2-ba4301ff8fe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:26.355636 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fc30b8-7e82-4418-bff2-ba4301ff8fe2 admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:26.355920 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fc30b8-7e82-4418-bff2-ba4301ff8fe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:26.356197 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fc30b8-7e82-4418-bff2-ba4301ff8fe2 admin admin] PATH_INFO: `/v3/users` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:26.356936 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fc30b8-7e82-4418-bff2-ba4301ff8fe2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:26.358918 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fc30b8-7e82-4418-bff2-ba4301ff8fe2 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:26.369146 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:17:26.436920 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1014/2000] 10.222.0.23 () {60 vars in 1143 bytes} [Sat Jun 18 05:17:26 2022] POST /identity/v3/users => generated 294 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:17:26.440991 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67425f19-fcce-4c19-96ec-e889786f5aef admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:26.451367 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67425f19-fcce-4c19-96ec-e889786f5aef admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:26.455975 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8381ca04-0b32-43a4-add2-74af6bcf358c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:26.456631 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8381ca04-0b32-43a4-add2-74af6bcf358c admin admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:26.456914 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8381ca04-0b32-43a4-add2-74af6bcf358c admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:26.457190 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8381ca04-0b32-43a4-add2-74af6bcf358c admin admin] PATH_INFO: `/v3/roles` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:26.457985 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8381ca04-0b32-43a4-add2-74af6bcf358c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:26.459826 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8381ca04-0b32-43a4-add2-74af6bcf358c admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:26.469777 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 987/2001] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:17:26 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:17:26.473598 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fc30b8-7e82-4418-bff2-ba4301ff8fe2 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:26.484469 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fc30b8-7e82-4418-bff2-ba4301ff8fe2 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:26.489276 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb969b00-f630-400b-86db-f185ad179891 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:26.489954 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb969b00-f630-400b-86db-f185ad179891 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:26.490239 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb969b00-f630-400b-86db-f185ad179891 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:26.490516 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb969b00-f630-400b-86db-f185ad179891 admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:26.492026 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb969b00-f630-400b-86db-f185ad179891 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:26.493652 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb969b00-f630-400b-86db-f185ad179891 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:26.503859 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1015/2002] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:17:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:17:26.508056 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8381ca04-0b32-43a4-add2-74af6bcf358c admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:26.518325 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8381ca04-0b32-43a4-add2-74af6bcf358c admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:26.523592 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0fcab3-8ec9-4449-b445-d28d58351ef1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:26.524734 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0fcab3-8ec9-4449-b445-d28d58351ef1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:26.525052 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0fcab3-8ec9-4449-b445-d28d58351ef1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:26.525327 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0fcab3-8ec9-4449-b445-d28d58351ef1 admin admin] PATH_INFO: `/v3/projects/e425c6ebf3c74a3faaf96aa04966de54/users/15864e93e68145dab0ba8487a763ba5e/roles/cbf5c4c26898413d93e73b46b41046a3` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:26.543500 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0fcab3-8ec9-4449-b445-d28d58351ef1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e425c6ebf3c74a3faaf96aa04966de54, user_id=15864e93e68145dab0ba8487a763ba5e, role_id=cbf5c4c26898413d93e73b46b41046a3)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:26.546438 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0fcab3-8ec9-4449-b445-d28d58351ef1 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:26.620256 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 988/2003] 10.222.0.23 () {60 vars in 1482 bytes} [Sat Jun 18 05:17:26 2022] PUT /identity/v3/projects/e425c6ebf3c74a3faaf96aa04966de54/users/15864e93e68145dab0ba8487a763ba5e/roles/cbf5c4c26898413d93e73b46b41046a3 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:17:26.625181 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:17:26.631150 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ea1cc0-caf7-4417-9173-8d17abda6bfc None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:26.631467 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ea1cc0-caf7-4417-9173-8d17abda6bfc None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:26.631956 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ea1cc0-caf7-4417-9173-8d17abda6bfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:26.658489 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-95ea1cc0-caf7-4417-9173-8d17abda6bfc None None] MFA Rules not processed for user `15864e93e68145dab0ba8487a763ba5e`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:17:26.659693 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-95ea1cc0-caf7-4417-9173-8d17abda6bfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:17:26.671298 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:17:26.728151 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1016/2004] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:17:26 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:17:26.729787 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:17:26.732507 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0fcab3-8ec9-4449-b445-d28d58351ef1 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:26.794464 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0fcab3-8ec9-4449-b445-d28d58351ef1 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:26.799162 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daad0479-c1ce-47fe-8932-1ba4ecd09c59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:26.799813 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daad0479-c1ce-47fe-8932-1ba4ecd09c59 admin admin] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:26.800102 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daad0479-c1ce-47fe-8932-1ba4ecd09c59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:26.800384 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daad0479-c1ce-47fe-8932-1ba4ecd09c59 admin admin] PATH_INFO: `/v3/projects` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:26.801104 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daad0479-c1ce-47fe-8932-1ba4ecd09c59 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:26.803028 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daad0479-c1ce-47fe-8932-1ba4ecd09c59 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:26.807143 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:17:26.808197 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:17:26.912495 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 989/2005] 10.222.0.23 () {60 vars in 1152 bytes} [Sat Jun 18 05:17:26 2022] POST /identity/v3/projects => generated 387 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:17:26.916881 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ea1cc0-caf7-4417-9173-8d17abda6bfc None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:26.929619 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ea1cc0-caf7-4417-9173-8d17abda6bfc None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:26.987149 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c85805-1603-4ed2-bd6a-72084baa6b42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:26.987854 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c85805-1603-4ed2-bd6a-72084baa6b42 admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:26.988137 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c85805-1603-4ed2-bd6a-72084baa6b42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:26.988414 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c85805-1603-4ed2-bd6a-72084baa6b42 admin admin] PATH_INFO: `/v3/users` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:26.989132 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c85805-1603-4ed2-bd6a-72084baa6b42 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:26.991077 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c85805-1603-4ed2-bd6a-72084baa6b42 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:27.083303 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1017/2006] 10.222.0.23 () {60 vars in 1143 bytes} [Sat Jun 18 05:17:26 2022] POST /identity/v3/users => generated 300 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:17:27.086782 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daad0479-c1ce-47fe-8932-1ba4ecd09c59 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:27.097910 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daad0479-c1ce-47fe-8932-1ba4ecd09c59 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:27.103433 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b95958-68bf-46e8-bb44-0b2b2b87bb8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:27.104087 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b95958-68bf-46e8-bb44-0b2b2b87bb8a admin admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:27.104369 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b95958-68bf-46e8-bb44-0b2b2b87bb8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:27.104646 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b95958-68bf-46e8-bb44-0b2b2b87bb8a admin admin] PATH_INFO: `/v3/roles` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:27.105366 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b95958-68bf-46e8-bb44-0b2b2b87bb8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:27.107009 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b95958-68bf-46e8-bb44-0b2b2b87bb8a admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:27.118909 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 990/2007] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:17:27 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:17:27.123039 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c85805-1603-4ed2-bd6a-72084baa6b42 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:27.133934 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c85805-1603-4ed2-bd6a-72084baa6b42 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:27.138688 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865c5628-5e74-41bd-bed9-f4bc3976ac52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:27.139578 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865c5628-5e74-41bd-bed9-f4bc3976ac52 admin admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:27.139863 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865c5628-5e74-41bd-bed9-f4bc3976ac52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:27.140142 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865c5628-5e74-41bd-bed9-f4bc3976ac52 admin admin] PATH_INFO: `/v3/projects/90556649c2bf4c96aebf24f7f7638a42/users/20290f20dbc54b339f3fe2ebaf34ec84/roles/21c65f2c51bd4dd283d13507d2714467` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:27.145252 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:17:27.158168 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865c5628-5e74-41bd-bed9-f4bc3976ac52 admin admin] RBAC: Authorizing `identity:create_grant(project_id=90556649c2bf4c96aebf24f7f7638a42, user_id=20290f20dbc54b339f3fe2ebaf34ec84, role_id=21c65f2c51bd4dd283d13507d2714467)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:27.160849 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865c5628-5e74-41bd-bed9-f4bc3976ac52 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:27.236553 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1018/2008] 10.222.0.23 () {60 vars in 1482 bytes} [Sat Jun 18 05:17:27 2022] PUT /identity/v3/projects/90556649c2bf4c96aebf24f7f7638a42/users/20290f20dbc54b339f3fe2ebaf34ec84/roles/21c65f2c51bd4dd283d13507d2714467 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:17:27.240392 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b95958-68bf-46e8-bb44-0b2b2b87bb8a admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:27.250721 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b95958-68bf-46e8-bb44-0b2b2b87bb8a admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:27.312624 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc76b413-d7a5-4176-9b4d-43435e17bce4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:27.313282 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc76b413-d7a5-4176-9b4d-43435e17bce4 admin admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:27.313598 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc76b413-d7a5-4176-9b4d-43435e17bce4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:27.313877 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc76b413-d7a5-4176-9b4d-43435e17bce4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:27.314600 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc76b413-d7a5-4176-9b4d-43435e17bce4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:27.316186 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc76b413-d7a5-4176-9b4d-43435e17bce4 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:27.326769 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 991/2009] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:17:27 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:17:27.330768 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865c5628-5e74-41bd-bed9-f4bc3976ac52 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:27.341685 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865c5628-5e74-41bd-bed9-f4bc3976ac52 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:27.346371 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19bb415-d283-4f9b-bf58-f4e62ccfe76d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:27.347256 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19bb415-d283-4f9b-bf58-f4e62ccfe76d admin admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:27.347535 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19bb415-d283-4f9b-bf58-f4e62ccfe76d admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:27.347806 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19bb415-d283-4f9b-bf58-f4e62ccfe76d admin admin] PATH_INFO: `/v3/projects/90556649c2bf4c96aebf24f7f7638a42/users/20290f20dbc54b339f3fe2ebaf34ec84/roles/21c65f2c51bd4dd283d13507d2714467` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:27.350786 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19bb415-d283-4f9b-bf58-f4e62ccfe76d admin admin] RBAC: Authorizing `identity:create_grant(project_id=90556649c2bf4c96aebf24f7f7638a42, user_id=20290f20dbc54b339f3fe2ebaf34ec84, role_id=21c65f2c51bd4dd283d13507d2714467)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:27.353858 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19bb415-d283-4f9b-bf58-f4e62ccfe76d admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:27.364232 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1019/2010] 10.222.0.23 () {60 vars in 1482 bytes} [Sat Jun 18 05:17:27 2022] PUT /identity/v3/projects/90556649c2bf4c96aebf24f7f7638a42/users/20290f20dbc54b339f3fe2ebaf34ec84/roles/21c65f2c51bd4dd283d13507d2714467 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 18 05:17:27.375416 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0524e306-90a1-4633-a931-cd0ac85c4e20 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:27.375711 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0524e306-90a1-4633-a931-cd0ac85c4e20 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:27.375995 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0524e306-90a1-4633-a931-cd0ac85c4e20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:27.404074 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-0524e306-90a1-4633-a931-cd0ac85c4e20 None None] MFA Rules not processed for user `20290f20dbc54b339f3fe2ebaf34ec84`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:17:27.405283 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-0524e306-90a1-4633-a931-cd0ac85c4e20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:17:27.475188 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 992/2011] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:17:27 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:17:27.487117 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19bb415-d283-4f9b-bf58-f4e62ccfe76d admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:27.488840 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-e19bb415-d283-4f9b-bf58-f4e62ccfe76d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:17:27.546998 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19bb415-d283-4f9b-bf58-f4e62ccfe76d admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:27.550658 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12bc3b1-2fa9-49a0-abff-77951163db55 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9de0428abc8400aa338aba4f7c34c94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:27.551296 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12bc3b1-2fa9-49a0-abff-77951163db55 service cinder] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:27.551579 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12bc3b1-2fa9-49a0-abff-77951163db55 service cinder] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:27.551854 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12bc3b1-2fa9-49a0-abff-77951163db55 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:27.561033 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12bc3b1-2fa9-49a0-abff-77951163db55 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:27.562508 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12bc3b1-2fa9-49a0-abff-77951163db55 service cinder] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:27.573267 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1020/2012] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:17:27 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:17:27.643079 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0524e306-90a1-4633-a931-cd0ac85c4e20 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:27.700708 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:17:27.704844 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0524e306-90a1-4633-a931-cd0ac85c4e20 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:27.709485 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a7548c-b1a5-4f09-a7a2-9a67646d91a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:27.711044 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a7548c-b1a5-4f09-a7a2-9a67646d91a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:27.711329 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a7548c-b1a5-4f09-a7a2-9a67646d91a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:27.711606 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a7548c-b1a5-4f09-a7a2-9a67646d91a0 admin admin] PATH_INFO: `/v3/users/15864e93e68145dab0ba8487a763ba5e` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:27.712896 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a7548c-b1a5-4f09-a7a2-9a67646d91a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=15864e93e68145dab0ba8487a763ba5e)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:27.714979 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a7548c-b1a5-4f09-a7a2-9a67646d91a0 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:27.879159 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-19a7548c-b1a5-4f09-a7a2-9a67646d91a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '15864e93e68145dab0ba8487a763ba5e'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:27.879761 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-19a7548c-b1a5-4f09-a7a2-9a67646d91a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '15864e93e68145dab0ba8487a763ba5e'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:27.973711 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-19a7548c-b1a5-4f09-a7a2-9a67646d91a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '15864e93e68145dab0ba8487a763ba5e'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:27.989566 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-19a7548c-b1a5-4f09-a7a2-9a67646d91a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '15864e93e68145dab0ba8487a763ba5e'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:27.997543 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-19a7548c-b1a5-4f09-a7a2-9a67646d91a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '15864e93e68145dab0ba8487a763ba5e'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:28.003698 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 993/2013] 10.222.0.23 () {58 vars in 1223 bytes} [Sat Jun 18 05:17:27 2022] DELETE /identity/v3/users/15864e93e68145dab0ba8487a763ba5e => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 18 05:17:28.009633 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef63822-0de9-4511-bcb3-1f4902b81f89 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:28.010181 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef63822-0de9-4511-bcb3-1f4902b81f89 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:28.010304 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef63822-0de9-4511-bcb3-1f4902b81f89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:28.030661 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-5ef63822-0de9-4511-bcb3-1f4902b81f89 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:17:28.031946 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-5ef63822-0de9-4511-bcb3-1f4902b81f89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:17:28.091750 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1021/2014] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:17:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:17:28.101419 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a7548c-b1a5-4f09-a7a2-9a67646d91a0 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:28.102956 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-19a7548c-b1a5-4f09-a7a2-9a67646d91a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:17:28.136447 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef63822-0de9-4511-bcb3-1f4902b81f89 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:28.138028 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-5ef63822-0de9-4511-bcb3-1f4902b81f89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:17:28.168967 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a7548c-b1a5-4f09-a7a2-9a67646d91a0 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:28.171840 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:17:28.173070 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aacbca1-1df7-4fde-b1d4-9e2461b0f90d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c4d7cde8f4ad9aa479bb09e325db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:28.173740 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aacbca1-1df7-4fde-b1d4-9e2461b0f90d service neutron] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:28.174022 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aacbca1-1df7-4fde-b1d4-9e2461b0f90d service neutron] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:28.174308 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aacbca1-1df7-4fde-b1d4-9e2461b0f90d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:28.184139 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aacbca1-1df7-4fde-b1d4-9e2461b0f90d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:28.185592 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aacbca1-1df7-4fde-b1d4-9e2461b0f90d service neutron] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:28.195881 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 994/2015] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:17:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 18 05:17:28.199268 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef63822-0de9-4511-bcb3-1f4902b81f89 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:28.203200 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec64aec-f8d7-4846-851b-e16b03b2d62f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9de0428abc8400aa338aba4f7c34c94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:28.203879 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec64aec-f8d7-4846-851b-e16b03b2d62f service cinder] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:28.204166 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec64aec-f8d7-4846-851b-e16b03b2d62f service cinder] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:28.204447 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec64aec-f8d7-4846-851b-e16b03b2d62f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:28.206511 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-7ec64aec-f8d7-4846-851b-e16b03b2d62f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:17:28.276434 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec64aec-f8d7-4846-851b-e16b03b2d62f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:28.279318 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec64aec-f8d7-4846-851b-e16b03b2d62f service cinder] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:28.289676 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1022/2016] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:17:28 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 18 05:17:28.643887 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aacbca1-1df7-4fde-b1d4-9e2461b0f90d service neutron] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:28.645518 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-1aacbca1-1df7-4fde-b1d4-9e2461b0f90d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:17:28.666589 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aacbca1-1df7-4fde-b1d4-9e2461b0f90d service neutron] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:28.672072 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5fd126-8969-48d2-a1d7-f0541271729d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:28.672953 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5fd126-8969-48d2-a1d7-f0541271729d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:28.673212 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5fd126-8969-48d2-a1d7-f0541271729d admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:28.673599 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5fd126-8969-48d2-a1d7-f0541271729d admin admin] PATH_INFO: `/v3/projects/e425c6ebf3c74a3faaf96aa04966de54` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:28.675832 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5fd126-8969-48d2-a1d7-f0541271729d admin admin] RBAC: Authorizing `identity:delete_project(project_id=e425c6ebf3c74a3faaf96aa04966de54)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:28.676869 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5fd126-8969-48d2-a1d7-f0541271729d admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:28.837798 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-ef5fd126-8969-48d2-a1d7-f0541271729d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e425c6ebf3c74a3faaf96aa04966de54', 'request_id': 'req-ef5fd126-8969-48d2-a1d7-f0541271729d'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:28.843926 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-ef5fd126-8969-48d2-a1d7-f0541271729d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e425c6ebf3c74a3faaf96aa04966de54', 'request_id': 'req-ef5fd126-8969-48d2-a1d7-f0541271729d'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:28.933646 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-ef5fd126-8969-48d2-a1d7-f0541271729d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e425c6ebf3c74a3faaf96aa04966de54', 'request_id': 'req-ef5fd126-8969-48d2-a1d7-f0541271729d'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:28.946410 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-ef5fd126-8969-48d2-a1d7-f0541271729d admin admin] The token cache is being invalidate because project e425c6ebf3c74a3faaf96aa04966de54 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80468) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 18 05:17:28.946703 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-ef5fd126-8969-48d2-a1d7-f0541271729d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:28.947383 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-ef5fd126-8969-48d2-a1d7-f0541271729d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:28.949008 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 995/2017] 10.222.0.23 () {58 vars in 1232 bytes} [Sat Jun 18 05:17:28 2022] DELETE /identity/v3/projects/e425c6ebf3c74a3faaf96aa04966de54 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:17:28.952976 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec64aec-f8d7-4846-851b-e16b03b2d62f service cinder] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:28.965325 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec64aec-f8d7-4846-851b-e16b03b2d62f service cinder] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:29.039644 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7842988f-f180-4582-9782-e96f1a9ed2d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:29.045625 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7842988f-f180-4582-9782-e96f1a9ed2d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:29.045625 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7842988f-f180-4582-9782-e96f1a9ed2d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:29.045625 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7842988f-f180-4582-9782-e96f1a9ed2d9 admin admin] PATH_INFO: `/v3/users/20290f20dbc54b339f3fe2ebaf34ec84` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:29.045625 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7842988f-f180-4582-9782-e96f1a9ed2d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=20290f20dbc54b339f3fe2ebaf34ec84)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:29.050932 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7842988f-f180-4582-9782-e96f1a9ed2d9 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:29.245300 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-7842988f-f180-4582-9782-e96f1a9ed2d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '20290f20dbc54b339f3fe2ebaf34ec84'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:29.257623 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-7842988f-f180-4582-9782-e96f1a9ed2d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '20290f20dbc54b339f3fe2ebaf34ec84'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:29.258598 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-7842988f-f180-4582-9782-e96f1a9ed2d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '20290f20dbc54b339f3fe2ebaf34ec84'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:29.322256 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-7842988f-f180-4582-9782-e96f1a9ed2d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '20290f20dbc54b339f3fe2ebaf34ec84'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:29.328281 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:17:29.330494 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-7842988f-f180-4582-9782-e96f1a9ed2d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '20290f20dbc54b339f3fe2ebaf34ec84'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:29.337028 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1023/2018] 10.222.0.23 () {58 vars in 1223 bytes} [Sat Jun 18 05:17:28 2022] DELETE /identity/v3/users/20290f20dbc54b339f3fe2ebaf34ec84 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:17:29.797857 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5fd126-8969-48d2-a1d7-f0541271729d admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:29.799280 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-ef5fd126-8969-48d2-a1d7-f0541271729d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:17:29.808749 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:17:29.819614 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5fd126-8969-48d2-a1d7-f0541271729d admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:29.877560 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bc3678-2ca1-4c80-8c8e-f4532029f8e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:29.878419 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bc3678-2ca1-4c80-8c8e-f4532029f8e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:29.878701 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bc3678-2ca1-4c80-8c8e-f4532029f8e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:29.878975 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bc3678-2ca1-4c80-8c8e-f4532029f8e7 admin admin] PATH_INFO: `/v3/projects/90556649c2bf4c96aebf24f7f7638a42` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:29.880155 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bc3678-2ca1-4c80-8c8e-f4532029f8e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=90556649c2bf4c96aebf24f7f7638a42)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:29.882212 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bc3678-2ca1-4c80-8c8e-f4532029f8e7 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:30.013612 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-56bc3678-2ca1-4c80-8c8e-f4532029f8e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90556649c2bf4c96aebf24f7f7638a42', 'request_id': 'req-56bc3678-2ca1-4c80-8c8e-f4532029f8e7'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:30.019273 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-56bc3678-2ca1-4c80-8c8e-f4532029f8e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90556649c2bf4c96aebf24f7f7638a42', 'request_id': 'req-56bc3678-2ca1-4c80-8c8e-f4532029f8e7'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:30.076006 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-56bc3678-2ca1-4c80-8c8e-f4532029f8e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90556649c2bf4c96aebf24f7f7638a42', 'request_id': 'req-56bc3678-2ca1-4c80-8c8e-f4532029f8e7'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:30.087899 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-56bc3678-2ca1-4c80-8c8e-f4532029f8e7 admin admin] The token cache is being invalidate because project 90556649c2bf4c96aebf24f7f7638a42 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80468) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 18 05:17:30.088185 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-56bc3678-2ca1-4c80-8c8e-f4532029f8e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:30.088810 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-56bc3678-2ca1-4c80-8c8e-f4532029f8e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:30.090395 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 996/2019] 10.222.0.23 () {58 vars in 1232 bytes} [Sat Jun 18 05:17:29 2022] DELETE /identity/v3/projects/90556649c2bf4c96aebf24f7f7638a42 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 18 05:17:30.095159 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e83a46-ea2c-4a6c-98ec-4835896bbb32 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:30.095644 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e83a46-ea2c-4a6c-98ec-4835896bbb32 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:30.095753 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e83a46-ea2c-4a6c-98ec-4835896bbb32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:30.113360 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-f8e83a46-ea2c-4a6c-98ec-4835896bbb32 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:17:30.114646 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-f8e83a46-ea2c-4a6c-98ec-4835896bbb32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:17:30.186108 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1024/2020] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:17:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:17:30.193765 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcfadd6-fe3a-4b98-9b69-eac553c91506 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:30.194059 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcfadd6-fe3a-4b98-9b69-eac553c91506 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:30.194346 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcfadd6-fe3a-4b98-9b69-eac553c91506 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:30.218073 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-7fcfadd6-fe3a-4b98-9b69-eac553c91506 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:17:30.219406 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-7fcfadd6-fe3a-4b98-9b69-eac553c91506 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:17:30.227611 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 997/2021] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:17:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:17:30.236097 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e83a46-ea2c-4a6c-98ec-4835896bbb32 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:30.246014 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e83a46-ea2c-4a6c-98ec-4835896bbb32 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:30.250889 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7937358b-c8c0-4cfa-9ed4-9a2e9a98faa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:30.251535 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7937358b-c8c0-4cfa-9ed4-9a2e9a98faa3 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:30.251814 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7937358b-c8c0-4cfa-9ed4-9a2e9a98faa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:30.252197 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7937358b-c8c0-4cfa-9ed4-9a2e9a98faa3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:30.252923 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7937358b-c8c0-4cfa-9ed4-9a2e9a98faa3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 05:17:30.253228 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7937358b-c8c0-4cfa-9ed4-9a2e9a98faa3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:30.254926 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7937358b-c8c0-4cfa-9ed4-9a2e9a98faa3 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:30.267151 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1025/2022] 10.222.0.23 () {58 vars in 1152 bytes} [Sat Jun 18 05:17:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 05:17:30.271415 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcfadd6-fe3a-4b98-9b69-eac553c91506 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:30.281894 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcfadd6-fe3a-4b98-9b69-eac553c91506 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:30.286674 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86e69ea-e02f-4ee2-9d49-40b974b333d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:30.287324 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86e69ea-e02f-4ee2-9d49-40b974b333d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:30.287611 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86e69ea-e02f-4ee2-9d49-40b974b333d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:30.287894 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86e69ea-e02f-4ee2-9d49-40b974b333d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:30.288607 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86e69ea-e02f-4ee2-9d49-40b974b333d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:30.290690 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86e69ea-e02f-4ee2-9d49-40b974b333d5 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:30.400439 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 998/2023] 10.222.0.23 () {60 vars in 1152 bytes} [Sat Jun 18 05:17:30 2022] POST /identity/v3/projects => generated 401 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:17:30.402945 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7937358b-c8c0-4cfa-9ed4-9a2e9a98faa3 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:30.413317 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7937358b-c8c0-4cfa-9ed4-9a2e9a98faa3 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:30.469993 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021db3d9-b478-4c1e-bbc6-f7d312647263 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:30.470676 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021db3d9-b478-4c1e-bbc6-f7d312647263 admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:30.470986 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021db3d9-b478-4c1e-bbc6-f7d312647263 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:30.471262 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021db3d9-b478-4c1e-bbc6-f7d312647263 admin admin] PATH_INFO: `/v3/users` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:30.471983 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021db3d9-b478-4c1e-bbc6-f7d312647263 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:30.473938 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021db3d9-b478-4c1e-bbc6-f7d312647263 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:30.545179 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1026/2024] 10.222.0.23 () {60 vars in 1143 bytes} [Sat Jun 18 05:17:30 2022] POST /identity/v3/users => generated 301 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:17:30.549544 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86e69ea-e02f-4ee2-9d49-40b974b333d5 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:30.561183 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86e69ea-e02f-4ee2-9d49-40b974b333d5 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:30.565874 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceca1efd-0742-4573-89e5-683a69fe31f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:30.566518 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceca1efd-0742-4573-89e5-683a69fe31f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:30.566804 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceca1efd-0742-4573-89e5-683a69fe31f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:30.567081 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceca1efd-0742-4573-89e5-683a69fe31f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:30.567832 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceca1efd-0742-4573-89e5-683a69fe31f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:30.569453 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceca1efd-0742-4573-89e5-683a69fe31f1 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:30.580963 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 999/2025] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:17:30 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:17:30.584910 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021db3d9-b478-4c1e-bbc6-f7d312647263 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:30.595311 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021db3d9-b478-4c1e-bbc6-f7d312647263 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:30.600018 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf97961c-7894-4700-a8ff-379a65db51d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:30.600662 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf97961c-7894-4700-a8ff-379a65db51d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:30.600945 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf97961c-7894-4700-a8ff-379a65db51d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:30.601220 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf97961c-7894-4700-a8ff-379a65db51d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:30.602311 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf97961c-7894-4700-a8ff-379a65db51d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:30.603751 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf97961c-7894-4700-a8ff-379a65db51d8 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:30.613870 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1027/2026] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:17:30 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:17:30.624806 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceca1efd-0742-4573-89e5-683a69fe31f1 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:30.636162 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceca1efd-0742-4573-89e5-683a69fe31f1 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:30.641106 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea7ca96-f16c-469c-ab09-d7590ece00d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:30.642033 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea7ca96-f16c-469c-ab09-d7590ece00d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:30.642311 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea7ca96-f16c-469c-ab09-d7590ece00d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:30.642583 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea7ca96-f16c-469c-ab09-d7590ece00d1 admin admin] PATH_INFO: `/v3/projects/a42be8af2a12411aa4c81096135870de/users/1407c649449f43cb8863b8a95493894f/roles/cbf5c4c26898413d93e73b46b41046a3` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:30.660304 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea7ca96-f16c-469c-ab09-d7590ece00d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a42be8af2a12411aa4c81096135870de, user_id=1407c649449f43cb8863b8a95493894f, role_id=cbf5c4c26898413d93e73b46b41046a3)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:30.663237 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea7ca96-f16c-469c-ab09-d7590ece00d1 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:30.750751 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1000/2027] 10.222.0.23 () {60 vars in 1482 bytes} [Sat Jun 18 05:17:30 2022] PUT /identity/v3/projects/a42be8af2a12411aa4c81096135870de/users/1407c649449f43cb8863b8a95493894f/roles/cbf5c4c26898413d93e73b46b41046a3 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:17:30.762426 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61f2c45-8de8-4b8e-826a-9d3dd4eee6ad None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:30.762714 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61f2c45-8de8-4b8e-826a-9d3dd4eee6ad None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:30.762993 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61f2c45-8de8-4b8e-826a-9d3dd4eee6ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:30.790405 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-b61f2c45-8de8-4b8e-826a-9d3dd4eee6ad None None] MFA Rules not processed for user `1407c649449f43cb8863b8a95493894f`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:17:30.791619 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-b61f2c45-8de8-4b8e-826a-9d3dd4eee6ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:17:30.840358 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:17:30.857687 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1028/2028] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:17:30 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:17:30.862141 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea7ca96-f16c-469c-ab09-d7590ece00d1 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:30.893965 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:17:30.926672 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea7ca96-f16c-469c-ab09-d7590ece00d1 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:30.932691 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfe626c-a226-4701-b98d-37cf3e127cff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:30.933336 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfe626c-a226-4701-b98d-37cf3e127cff admin admin] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:30.933649 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfe626c-a226-4701-b98d-37cf3e127cff admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:30.933923 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfe626c-a226-4701-b98d-37cf3e127cff admin admin] PATH_INFO: `/v3/projects` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:30.934607 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfe626c-a226-4701-b98d-37cf3e127cff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:30.936574 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfe626c-a226-4701-b98d-37cf3e127cff admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:31.013620 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1001/2029] 10.222.0.23 () {60 vars in 1152 bytes} [Sat Jun 18 05:17:30 2022] POST /identity/v3/projects => generated 399 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:17:31.017510 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61f2c45-8de8-4b8e-826a-9d3dd4eee6ad None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:31.028603 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61f2c45-8de8-4b8e-826a-9d3dd4eee6ad None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:31.084069 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9339360b-a7d9-4e42-94b2-893eb9633918 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:31.084757 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9339360b-a7d9-4e42-94b2-893eb9633918 admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:31.085041 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9339360b-a7d9-4e42-94b2-893eb9633918 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:31.085320 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9339360b-a7d9-4e42-94b2-893eb9633918 admin admin] PATH_INFO: `/v3/users` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:31.086134 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9339360b-a7d9-4e42-94b2-893eb9633918 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:31.088063 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9339360b-a7d9-4e42-94b2-893eb9633918 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:31.199073 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1029/2030] 10.222.0.23 () {60 vars in 1143 bytes} [Sat Jun 18 05:17:31 2022] POST /identity/v3/users => generated 306 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:17:31.207890 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfe626c-a226-4701-b98d-37cf3e127cff admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:31.220151 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfe626c-a226-4701-b98d-37cf3e127cff admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:31.225072 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd02c5fd-fa67-4696-af58-1cac4c0efe0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:31.225837 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd02c5fd-fa67-4696-af58-1cac4c0efe0c admin admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:31.226253 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd02c5fd-fa67-4696-af58-1cac4c0efe0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:31.227488 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd02c5fd-fa67-4696-af58-1cac4c0efe0c admin admin] PATH_INFO: `/v3/roles` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:31.228423 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd02c5fd-fa67-4696-af58-1cac4c0efe0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:31.230153 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd02c5fd-fa67-4696-af58-1cac4c0efe0c admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:31.243082 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1002/2031] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:17:31 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:17:31.247040 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9339360b-a7d9-4e42-94b2-893eb9633918 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:31.257639 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9339360b-a7d9-4e42-94b2-893eb9633918 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:31.262392 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b73275b-f85c-4f17-bfe9-0e281a7e9da9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:31.263275 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b73275b-f85c-4f17-bfe9-0e281a7e9da9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:31.263561 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b73275b-f85c-4f17-bfe9-0e281a7e9da9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:31.263839 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b73275b-f85c-4f17-bfe9-0e281a7e9da9 admin admin] PATH_INFO: `/v3/projects/fa46c48e05f24c3693a146fcf47cb108/users/1357b47593e240e1842a97f004c49e8f/roles/21c65f2c51bd4dd283d13507d2714467` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:31.279810 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b73275b-f85c-4f17-bfe9-0e281a7e9da9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa46c48e05f24c3693a146fcf47cb108, user_id=1357b47593e240e1842a97f004c49e8f, role_id=21c65f2c51bd4dd283d13507d2714467)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:31.282550 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b73275b-f85c-4f17-bfe9-0e281a7e9da9 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:31.340060 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1030/2032] 10.222.0.23 () {60 vars in 1482 bytes} [Sat Jun 18 05:17:31 2022] PUT /identity/v3/projects/fa46c48e05f24c3693a146fcf47cb108/users/1357b47593e240e1842a97f004c49e8f/roles/21c65f2c51bd4dd283d13507d2714467 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:17:31.343995 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd02c5fd-fa67-4696-af58-1cac4c0efe0c admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:31.355269 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd02c5fd-fa67-4696-af58-1cac4c0efe0c admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:31.412846 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345a6156-f4d6-4613-81be-b4487360afa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:31.413555 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345a6156-f4d6-4613-81be-b4487360afa7 admin admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:31.413842 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345a6156-f4d6-4613-81be-b4487360afa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:31.414704 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345a6156-f4d6-4613-81be-b4487360afa7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:31.415604 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345a6156-f4d6-4613-81be-b4487360afa7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:31.417208 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345a6156-f4d6-4613-81be-b4487360afa7 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:31.426930 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1003/2033] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:17:31 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:17:31.431218 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b73275b-f85c-4f17-bfe9-0e281a7e9da9 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:31.441646 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b73275b-f85c-4f17-bfe9-0e281a7e9da9 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:31.446297 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b7858f-994b-4a02-84d7-1e5b8b215aeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:31.447163 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b7858f-994b-4a02-84d7-1e5b8b215aeb admin admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:31.447452 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b7858f-994b-4a02-84d7-1e5b8b215aeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:31.447733 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b7858f-994b-4a02-84d7-1e5b8b215aeb admin admin] PATH_INFO: `/v3/projects/fa46c48e05f24c3693a146fcf47cb108/users/1357b47593e240e1842a97f004c49e8f/roles/21c65f2c51bd4dd283d13507d2714467` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:31.449733 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b7858f-994b-4a02-84d7-1e5b8b215aeb admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa46c48e05f24c3693a146fcf47cb108, user_id=1357b47593e240e1842a97f004c49e8f, role_id=21c65f2c51bd4dd283d13507d2714467)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:31.452442 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b7858f-994b-4a02-84d7-1e5b8b215aeb admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:31.463075 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1031/2034] 10.222.0.23 () {60 vars in 1482 bytes} [Sat Jun 18 05:17:31 2022] PUT /identity/v3/projects/fa46c48e05f24c3693a146fcf47cb108/users/1357b47593e240e1842a97f004c49e8f/roles/21c65f2c51bd4dd283d13507d2714467 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:17:31.472755 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e44f67b-6433-4255-9156-8286a75b9347 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:31.473049 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e44f67b-6433-4255-9156-8286a75b9347 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:31.473331 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e44f67b-6433-4255-9156-8286a75b9347 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:31.499700 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-9e44f67b-6433-4255-9156-8286a75b9347 None None] MFA Rules not processed for user `1357b47593e240e1842a97f004c49e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:17:31.500915 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-9e44f67b-6433-4255-9156-8286a75b9347 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:17:31.568468 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1004/2035] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:17:31 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:17:31.579877 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b7858f-994b-4a02-84d7-1e5b8b215aeb admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:31.581611 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-75b7858f-994b-4a02-84d7-1e5b8b215aeb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:17:31.639077 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b7858f-994b-4a02-84d7-1e5b8b215aeb admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:31.642900 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c88023c-fb76-4480-bf4f-88929f6424e0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9de0428abc8400aa338aba4f7c34c94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:31.643535 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c88023c-fb76-4480-bf4f-88929f6424e0 service cinder] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:31.643822 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c88023c-fb76-4480-bf4f-88929f6424e0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:31.644117 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c88023c-fb76-4480-bf4f-88929f6424e0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:31.654014 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c88023c-fb76-4480-bf4f-88929f6424e0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:31.655453 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c88023c-fb76-4480-bf4f-88929f6424e0 service cinder] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:31.665008 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1032/2036] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:17:31 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:17:31.832419 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e44f67b-6433-4255-9156-8286a75b9347 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:31.843550 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e44f67b-6433-4255-9156-8286a75b9347 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:31.847271 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d97bef-1f9e-40d4-9ccb-7c5d652c643a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9de0428abc8400aa338aba4f7c34c94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:31.848334 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d97bef-1f9e-40d4-9ccb-7c5d652c643a service cinder] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:31.848864 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d97bef-1f9e-40d4-9ccb-7c5d652c643a service cinder] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:31.849368 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d97bef-1f9e-40d4-9ccb-7c5d652c643a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:31.874325 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:17:31.911067 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d97bef-1f9e-40d4-9ccb-7c5d652c643a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:31.912513 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d97bef-1f9e-40d4-9ccb-7c5d652c643a service cinder] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:31.921911 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1005/2037] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:17:31 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:17:40.573495 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c88023c-fb76-4480-bf4f-88929f6424e0 service cinder] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:40.678246 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:17:40.694560 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c88023c-fb76-4480-bf4f-88929f6424e0 service cinder] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:40.699531 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3366c289-fb63-43cd-9bba-8da5e3fd8e15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:40.700384 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3366c289-fb63-43cd-9bba-8da5e3fd8e15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:40.700674 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3366c289-fb63-43cd-9bba-8da5e3fd8e15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:40.700952 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3366c289-fb63-43cd-9bba-8da5e3fd8e15 admin admin] PATH_INFO: `/v3/users/1407c649449f43cb8863b8a95493894f` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:40.702249 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3366c289-fb63-43cd-9bba-8da5e3fd8e15 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1407c649449f43cb8863b8a95493894f)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:40.704182 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3366c289-fb63-43cd-9bba-8da5e3fd8e15 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:40.939798 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-3366c289-fb63-43cd-9bba-8da5e3fd8e15 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1407c649449f43cb8863b8a95493894f'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:40.955157 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-3366c289-fb63-43cd-9bba-8da5e3fd8e15 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1407c649449f43cb8863b8a95493894f'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:40.956020 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-3366c289-fb63-43cd-9bba-8da5e3fd8e15 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1407c649449f43cb8863b8a95493894f'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:41.030796 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-3366c289-fb63-43cd-9bba-8da5e3fd8e15 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1407c649449f43cb8863b8a95493894f'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:41.040664 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-3366c289-fb63-43cd-9bba-8da5e3fd8e15 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1407c649449f43cb8863b8a95493894f'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:41.047131 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1033/2038] 10.222.0.23 () {58 vars in 1223 bytes} [Sat Jun 18 05:17:40 2022] DELETE /identity/v3/users/1407c649449f43cb8863b8a95493894f => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 18 05:17:41.051759 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5dd200-05c4-40b7-a5ef-d336c426c932 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:41.052318 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5dd200-05c4-40b7-a5ef-d336c426c932 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:41.052688 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5dd200-05c4-40b7-a5ef-d336c426c932 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:41.071640 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-4c5dd200-05c4-40b7-a5ef-d336c426c932 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:17:41.073272 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-4c5dd200-05c4-40b7-a5ef-d336c426c932 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:17:41.148013 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1006/2039] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:17:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 18 05:17:41.148395 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9860e3-8d1e-4d4a-92f1-06b38b567140 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:41.148765 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9860e3-8d1e-4d4a-92f1-06b38b567140 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:41.149652 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9860e3-8d1e-4d4a-92f1-06b38b567140 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:41.151415 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1034/2040] 10.222.0.23 () {54 vars in 921 bytes} [Sat Jun 18 05:17:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 18 05:17:41.159052 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9860e3-8d1e-4d4a-92f1-06b38b567140 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:41.161495 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e372fc3b-16cb-4d23-8b56-aacf3a93e4b6 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:41.167628 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e372fc3b-16cb-4d23-8b56-aacf3a93e4b6 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:41.167628 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e372fc3b-16cb-4d23-8b56-aacf3a93e4b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:41.167842 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-df9860e3-8d1e-4d4a-92f1-06b38b567140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:17:41.181531 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-e372fc3b-16cb-4d23-8b56-aacf3a93e4b6 None None] MFA Rules not processed for user `5323ffd91f864680b0c0084eb3f44d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:17:41.182767 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-e372fc3b-16cb-4d23-8b56-aacf3a93e4b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:17:41.252512 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9860e3-8d1e-4d4a-92f1-06b38b567140 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:41.257618 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bb003d-9912-4fd5-8766-9fa0815c15ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c4d7cde8f4ad9aa479bb09e325db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:41.258268 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bb003d-9912-4fd5-8766-9fa0815c15ca service neutron] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:41.259263 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bb003d-9912-4fd5-8766-9fa0815c15ca service neutron] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:41.259547 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bb003d-9912-4fd5-8766-9fa0815c15ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:41.272064 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bb003d-9912-4fd5-8766-9fa0815c15ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:41.273785 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bb003d-9912-4fd5-8766-9fa0815c15ca service neutron] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:41.289289 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1035/2041] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:17:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:17:41.292729 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1007/2042] 10.222.0.23 () {58 vars in 1018 bytes} [Sat Jun 18 05:17:41 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:17:41.319177 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e372fc3b-16cb-4d23-8b56-aacf3a93e4b6 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:41.323210 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-e372fc3b-16cb-4d23-8b56-aacf3a93e4b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:17:41.345682 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e372fc3b-16cb-4d23-8b56-aacf3a93e4b6 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:41.359013 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deca6a77-aebf-4a9c-9842-b650a5dac738 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5323ffd91f864680b0c0084eb3f44d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:41.360203 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deca6a77-aebf-4a9c-9842-b650a5dac738 service placement] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:41.364558 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deca6a77-aebf-4a9c-9842-b650a5dac738 service placement] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:41.364558 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deca6a77-aebf-4a9c-9842-b650a5dac738 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:41.395980 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deca6a77-aebf-4a9c-9842-b650a5dac738 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:41.400291 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deca6a77-aebf-4a9c-9842-b650a5dac738 service placement] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:41.418631 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1008/2043] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:17:41 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:17:41.940518 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bb003d-9912-4fd5-8766-9fa0815c15ca service neutron] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:41.942210 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-a5bb003d-9912-4fd5-8766-9fa0815c15ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:17:41.972722 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bb003d-9912-4fd5-8766-9fa0815c15ca service neutron] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:41.980017 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc61db1e-289a-4cee-85d1-2ac119487bba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:41.981315 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc61db1e-289a-4cee-85d1-2ac119487bba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:41.981933 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc61db1e-289a-4cee-85d1-2ac119487bba admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:41.982407 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc61db1e-289a-4cee-85d1-2ac119487bba admin admin] PATH_INFO: `/v3/projects/a42be8af2a12411aa4c81096135870de` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:41.983879 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc61db1e-289a-4cee-85d1-2ac119487bba admin admin] RBAC: Authorizing `identity:delete_project(project_id=a42be8af2a12411aa4c81096135870de)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:41.986024 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc61db1e-289a-4cee-85d1-2ac119487bba admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:42.088211 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-fc61db1e-289a-4cee-85d1-2ac119487bba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a42be8af2a12411aa4c81096135870de', 'request_id': 'req-fc61db1e-289a-4cee-85d1-2ac119487bba'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:42.094071 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-fc61db1e-289a-4cee-85d1-2ac119487bba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a42be8af2a12411aa4c81096135870de', 'request_id': 'req-fc61db1e-289a-4cee-85d1-2ac119487bba'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:42.100192 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-fc61db1e-289a-4cee-85d1-2ac119487bba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a42be8af2a12411aa4c81096135870de', 'request_id': 'req-fc61db1e-289a-4cee-85d1-2ac119487bba'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:42.101896 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:17:42.134644 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deca6a77-aebf-4a9c-9842-b650a5dac738 service placement] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:42.136059 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-deca6a77-aebf-4a9c-9842-b650a5dac738 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:17:42.155278 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deca6a77-aebf-4a9c-9842-b650a5dac738 service placement] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:42.185848 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-fc61db1e-289a-4cee-85d1-2ac119487bba admin admin] The token cache is being invalidate because project a42be8af2a12411aa4c81096135870de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80469) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 18 05:17:42.185848 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-fc61db1e-289a-4cee-85d1-2ac119487bba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:42.186435 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-fc61db1e-289a-4cee-85d1-2ac119487bba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:42.188008 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1036/2044] 10.222.0.23 () {58 vars in 1232 bytes} [Sat Jun 18 05:17:41 2022] DELETE /identity/v3/projects/a42be8af2a12411aa4c81096135870de => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:17:42.191929 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc61db1e-289a-4cee-85d1-2ac119487bba admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:42.193236 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-fc61db1e-289a-4cee-85d1-2ac119487bba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:17:42.202683 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:17:42.215474 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7c6451-40f5-4cf6-b867-1c3b6e66f8db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:42.216571 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7c6451-40f5-4cf6-b867-1c3b6e66f8db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:42.216850 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7c6451-40f5-4cf6-b867-1c3b6e66f8db admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:42.217121 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7c6451-40f5-4cf6-b867-1c3b6e66f8db admin admin] PATH_INFO: `/v3/users/331d59a2875741838b6502c3cea39bd8` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:42.218501 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7c6451-40f5-4cf6-b867-1c3b6e66f8db admin admin] RBAC: Authorizing `identity:delete_user(user_id=331d59a2875741838b6502c3cea39bd8)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:42.220440 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7c6451-40f5-4cf6-b867-1c3b6e66f8db admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:42.258468 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc61db1e-289a-4cee-85d1-2ac119487bba admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:42.263542 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be09c5c-4748-4dc6-9d28-6e2e6908edab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:42.264405 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be09c5c-4748-4dc6-9d28-6e2e6908edab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:42.264692 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be09c5c-4748-4dc6-9d28-6e2e6908edab admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:42.264998 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be09c5c-4748-4dc6-9d28-6e2e6908edab admin admin] PATH_INFO: `/v3/users/1357b47593e240e1842a97f004c49e8f` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:42.266243 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be09c5c-4748-4dc6-9d28-6e2e6908edab admin admin] RBAC: Authorizing `identity:delete_user(user_id=1357b47593e240e1842a97f004c49e8f)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:42.268191 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be09c5c-4748-4dc6-9d28-6e2e6908edab admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:42.365128 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-8e7c6451-40f5-4cf6-b867-1c3b6e66f8db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '331d59a2875741838b6502c3cea39bd8'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:42.365806 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-8e7c6451-40f5-4cf6-b867-1c3b6e66f8db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '331d59a2875741838b6502c3cea39bd8'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:42.413645 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-8e7c6451-40f5-4cf6-b867-1c3b6e66f8db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '331d59a2875741838b6502c3cea39bd8'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:42.427780 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-8e7c6451-40f5-4cf6-b867-1c3b6e66f8db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '331d59a2875741838b6502c3cea39bd8'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:42.436172 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-8e7c6451-40f5-4cf6-b867-1c3b6e66f8db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '331d59a2875741838b6502c3cea39bd8'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:42.446926 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-9be09c5c-4748-4dc6-9d28-6e2e6908edab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1357b47593e240e1842a97f004c49e8f'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:42.452120 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1009/2045] 10.222.0.23 () {58 vars in 1223 bytes} [Sat Jun 18 05:17:42 2022] DELETE /identity/v3/users/331d59a2875741838b6502c3cea39bd8 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:17:42.459439 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-9be09c5c-4748-4dc6-9d28-6e2e6908edab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1357b47593e240e1842a97f004c49e8f'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:42.460289 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-9be09c5c-4748-4dc6-9d28-6e2e6908edab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1357b47593e240e1842a97f004c49e8f'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:42.504987 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-9be09c5c-4748-4dc6-9d28-6e2e6908edab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1357b47593e240e1842a97f004c49e8f'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:42.514597 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-9be09c5c-4748-4dc6-9d28-6e2e6908edab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1357b47593e240e1842a97f004c49e8f'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:42.520930 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1037/2046] 10.222.0.23 () {58 vars in 1223 bytes} [Sat Jun 18 05:17:42 2022] DELETE /identity/v3/users/1357b47593e240e1842a97f004c49e8f => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:17:42.723653 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7c6451-40f5-4cf6-b867-1c3b6e66f8db admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:42.725091 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-8e7c6451-40f5-4cf6-b867-1c3b6e66f8db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:17:42.813472 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7c6451-40f5-4cf6-b867-1c3b6e66f8db admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:42.877717 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3473d975-eb57-4e05-9b23-b1c2416710dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:42.878580 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3473d975-eb57-4e05-9b23-b1c2416710dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:42.878867 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3473d975-eb57-4e05-9b23-b1c2416710dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:42.879146 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3473d975-eb57-4e05-9b23-b1c2416710dc admin admin] PATH_INFO: `/v3/projects/ceb4ebf6b13149448546688ff0599c6c` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:42.881022 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3473d975-eb57-4e05-9b23-b1c2416710dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=ceb4ebf6b13149448546688ff0599c6c)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:42.883045 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3473d975-eb57-4e05-9b23-b1c2416710dc admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:42.901540 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:17:42.986653 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be09c5c-4748-4dc6-9d28-6e2e6908edab admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:42.988700 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-9be09c5c-4748-4dc6-9d28-6e2e6908edab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:17:42.999828 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-3473d975-eb57-4e05-9b23-b1c2416710dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ceb4ebf6b13149448546688ff0599c6c', 'request_id': 'req-3473d975-eb57-4e05-9b23-b1c2416710dc'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:43.005711 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-3473d975-eb57-4e05-9b23-b1c2416710dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ceb4ebf6b13149448546688ff0599c6c', 'request_id': 'req-3473d975-eb57-4e05-9b23-b1c2416710dc'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:43.006837 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be09c5c-4748-4dc6-9d28-6e2e6908edab admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:43.059056 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-3473d975-eb57-4e05-9b23-b1c2416710dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ceb4ebf6b13149448546688ff0599c6c', 'request_id': 'req-3473d975-eb57-4e05-9b23-b1c2416710dc'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:43.069177 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928883da-e64c-48df-8f09-c70ad81025f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:43.070137 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928883da-e64c-48df-8f09-c70ad81025f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:43.070420 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928883da-e64c-48df-8f09-c70ad81025f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:43.070692 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928883da-e64c-48df-8f09-c70ad81025f6 admin admin] PATH_INFO: `/v3/projects/fa46c48e05f24c3693a146fcf47cb108` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:43.070787 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-3473d975-eb57-4e05-9b23-b1c2416710dc admin admin] The token cache is being invalidate because project ceb4ebf6b13149448546688ff0599c6c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80468) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 18 05:17:43.071630 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-3473d975-eb57-4e05-9b23-b1c2416710dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:43.072396 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-3473d975-eb57-4e05-9b23-b1c2416710dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:43.072754 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928883da-e64c-48df-8f09-c70ad81025f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa46c48e05f24c3693a146fcf47cb108)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:43.074214 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1010/2047] 10.222.0.23 () {58 vars in 1232 bytes} [Sat Jun 18 05:17:42 2022] DELETE /identity/v3/projects/ceb4ebf6b13149448546688ff0599c6c => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:17:43.074658 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928883da-e64c-48df-8f09-c70ad81025f6 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:43.077821 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3473d975-eb57-4e05-9b23-b1c2416710dc admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:43.079166 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-3473d975-eb57-4e05-9b23-b1c2416710dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:17:43.087147 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:INSERT=2,keystone:DELETE=11,keystone:SELECT=87 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:17:43.091512 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3473d975-eb57-4e05-9b23-b1c2416710dc admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:43.096318 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fa1cf4-16ca-4e1d-9dfb-23985c4465b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:43.097183 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fa1cf4-16ca-4e1d-9dfb-23985c4465b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:43.097497 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fa1cf4-16ca-4e1d-9dfb-23985c4465b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:43.097777 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fa1cf4-16ca-4e1d-9dfb-23985c4465b9 admin admin] PATH_INFO: `/v3/users/c7ba925fddb34ddc96af30fc06271861` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:43.098967 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fa1cf4-16ca-4e1d-9dfb-23985c4465b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c7ba925fddb34ddc96af30fc06271861)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:43.100896 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fa1cf4-16ca-4e1d-9dfb-23985c4465b9 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:43.156069 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-928883da-e64c-48df-8f09-c70ad81025f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa46c48e05f24c3693a146fcf47cb108', 'request_id': 'req-928883da-e64c-48df-8f09-c70ad81025f6'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:43.161489 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-928883da-e64c-48df-8f09-c70ad81025f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa46c48e05f24c3693a146fcf47cb108', 'request_id': 'req-928883da-e64c-48df-8f09-c70ad81025f6'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:43.167862 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-928883da-e64c-48df-8f09-c70ad81025f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa46c48e05f24c3693a146fcf47cb108', 'request_id': 'req-928883da-e64c-48df-8f09-c70ad81025f6'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:43.281794 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-928883da-e64c-48df-8f09-c70ad81025f6 admin admin] The token cache is being invalidate because project fa46c48e05f24c3693a146fcf47cb108 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80469) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 18 05:17:43.282000 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-928883da-e64c-48df-8f09-c70ad81025f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:43.282612 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-928883da-e64c-48df-8f09-c70ad81025f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:43.284161 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1038/2048] 10.222.0.23 () {58 vars in 1232 bytes} [Sat Jun 18 05:17:42 2022] DELETE /identity/v3/projects/fa46c48e05f24c3693a146fcf47cb108 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:17:43.289622 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec5aa0d-4f60-4ace-8468-7018bdbe89f8 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:43.289944 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec5aa0d-4f60-4ace-8468-7018bdbe89f8 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:43.290210 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec5aa0d-4f60-4ace-8468-7018bdbe89f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:43.315702 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-bec5aa0d-4f60-4ace-8468-7018bdbe89f8 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:17:43.317497 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-bec5aa0d-4f60-4ace-8468-7018bdbe89f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:17:43.320275 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-51fa1cf4-16ca-4e1d-9dfb-23985c4465b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c7ba925fddb34ddc96af30fc06271861'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:43.321657 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-51fa1cf4-16ca-4e1d-9dfb-23985c4465b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c7ba925fddb34ddc96af30fc06271861'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:43.384121 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1039/2049] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:17:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 18 05:17:43.391812 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-51fa1cf4-16ca-4e1d-9dfb-23985c4465b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c7ba925fddb34ddc96af30fc06271861'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:43.392567 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cf870a-a259-41df-ab86-d993b1925eb8 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:43.392992 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cf870a-a259-41df-ab86-d993b1925eb8 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:43.393647 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cf870a-a259-41df-ab86-d993b1925eb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:43.405802 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-51fa1cf4-16ca-4e1d-9dfb-23985c4465b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c7ba925fddb34ddc96af30fc06271861'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:43.414466 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-42cf870a-a259-41df-ab86-d993b1925eb8 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:17:43.415377 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-51fa1cf4-16ca-4e1d-9dfb-23985c4465b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c7ba925fddb34ddc96af30fc06271861'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:43.415915 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-42cf870a-a259-41df-ab86-d993b1925eb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:17:43.422264 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1011/2050] 10.222.0.23 () {58 vars in 1223 bytes} [Sat Jun 18 05:17:43 2022] DELETE /identity/v3/users/c7ba925fddb34ddc96af30fc06271861 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 18 05:17:43.427198 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1040/2051] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:17:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 18 05:17:43.432402 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fa1cf4-16ca-4e1d-9dfb-23985c4465b9 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:43.519861 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fa1cf4-16ca-4e1d-9dfb-23985c4465b9 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:43.525993 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f66f60-474c-403f-8651-7d505df0c953 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:43.526659 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f66f60-474c-403f-8651-7d505df0c953 admin admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:43.526953 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f66f60-474c-403f-8651-7d505df0c953 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:43.527228 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f66f60-474c-403f-8651-7d505df0c953 admin admin] PATH_INFO: `/v3/domains` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:43.527965 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f66f60-474c-403f-8651-7d505df0c953 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 05:17:43.528279 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f66f60-474c-403f-8651-7d505df0c953 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:43.529981 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f66f60-474c-403f-8651-7d505df0c953 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:43.546743 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1012/2052] 10.222.0.23 () {58 vars in 1152 bytes} [Sat Jun 18 05:17:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 05:17:43.547807 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cf870a-a259-41df-ab86-d993b1925eb8 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:43.557879 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cf870a-a259-41df-ab86-d993b1925eb8 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:43.562828 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fc40ef-14e9-4f4c-bca6-39e9a5e5fff1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:43.563490 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fc40ef-14e9-4f4c-bca6-39e9a5e5fff1 admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:43.563835 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fc40ef-14e9-4f4c-bca6-39e9a5e5fff1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:43.564036 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fc40ef-14e9-4f4c-bca6-39e9a5e5fff1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:43.564748 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fc40ef-14e9-4f4c-bca6-39e9a5e5fff1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:43.566748 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fc40ef-14e9-4f4c-bca6-39e9a5e5fff1 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:43.658451 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1041/2053] 10.222.0.23 () {60 vars in 1152 bytes} [Sat Jun 18 05:17:43 2022] POST /identity/v3/projects => generated 375 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 18 05:17:43.662575 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f66f60-474c-403f-8651-7d505df0c953 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:43.673254 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f66f60-474c-403f-8651-7d505df0c953 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:43.677904 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:17:43.705994 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fc40ef-14e9-4f4c-bca6-39e9a5e5fff1 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:43.708939 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-91fc40ef-14e9-4f4c-bca6-39e9a5e5fff1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:17:43.751678 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08395424-bd40-4505-b469-6f7940695a1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:43.753615 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08395424-bd40-4505-b469-6f7940695a1a admin admin] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:43.753615 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08395424-bd40-4505-b469-6f7940695a1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:43.753615 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08395424-bd40-4505-b469-6f7940695a1a admin admin] PATH_INFO: `/v3/users` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:43.754434 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08395424-bd40-4505-b469-6f7940695a1a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:43.756838 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08395424-bd40-4505-b469-6f7940695a1a admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:43.803767 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fc40ef-14e9-4f4c-bca6-39e9a5e5fff1 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:43.809671 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7812f8bf-8d72-4f8d-a17f-1b86193a8e2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:43.810541 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7812f8bf-8d72-4f8d-a17f-1b86193a8e2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:43.810897 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7812f8bf-8d72-4f8d-a17f-1b86193a8e2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:43.811097 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7812f8bf-8d72-4f8d-a17f-1b86193a8e2c admin admin] PATH_INFO: `/v3/projects/9e7df6c98bc241398fb734d693186ca0` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:43.812411 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7812f8bf-8d72-4f8d-a17f-1b86193a8e2c admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e7df6c98bc241398fb734d693186ca0)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:43.814383 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7812f8bf-8d72-4f8d-a17f-1b86193a8e2c admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:43.834076 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:17:43.859666 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1013/2054] 10.222.0.23 () {60 vars in 1143 bytes} [Sat Jun 18 05:17:43 2022] POST /identity/v3/users => generated 288 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:17:43.876525 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08395424-bd40-4505-b469-6f7940695a1a admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:43.888156 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08395424-bd40-4505-b469-6f7940695a1a admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:43.893652 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407b3a11-e217-409f-b1c7-9306104993c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:43.894556 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407b3a11-e217-409f-b1c7-9306104993c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:43.894819 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407b3a11-e217-409f-b1c7-9306104993c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:43.895097 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407b3a11-e217-409f-b1c7-9306104993c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:43.895870 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407b3a11-e217-409f-b1c7-9306104993c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:43.897664 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407b3a11-e217-409f-b1c7-9306104993c3 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:43.911201 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1014/2055] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:17:43 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:17:43.915594 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407b3a11-e217-409f-b1c7-9306104993c3 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:43.926324 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407b3a11-e217-409f-b1c7-9306104993c3 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:43.932099 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a696b7-45c3-4e22-a764-46b391e39882 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:43.932802 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a696b7-45c3-4e22-a764-46b391e39882 admin admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:43.933163 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a696b7-45c3-4e22-a764-46b391e39882 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:43.933546 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a696b7-45c3-4e22-a764-46b391e39882 admin admin] PATH_INFO: `/v3/roles` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:43.934350 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a696b7-45c3-4e22-a764-46b391e39882 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:43.935964 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a696b7-45c3-4e22-a764-46b391e39882 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:43.947081 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-7812f8bf-8d72-4f8d-a17f-1b86193a8e2c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e7df6c98bc241398fb734d693186ca0', 'request_id': 'req-7812f8bf-8d72-4f8d-a17f-1b86193a8e2c'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:43.947278 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1015/2056] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:17:43 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 18 05:17:43.951419 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a696b7-45c3-4e22-a764-46b391e39882 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:43.955879 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-7812f8bf-8d72-4f8d-a17f-1b86193a8e2c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e7df6c98bc241398fb734d693186ca0', 'request_id': 'req-7812f8bf-8d72-4f8d-a17f-1b86193a8e2c'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:43.962237 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a696b7-45c3-4e22-a764-46b391e39882 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:43.964790 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-7812f8bf-8d72-4f8d-a17f-1b86193a8e2c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e7df6c98bc241398fb734d693186ca0', 'request_id': 'req-7812f8bf-8d72-4f8d-a17f-1b86193a8e2c'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:44.019880 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78341bb-8a1c-4f9c-8c7c-64f9bf3e1343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:44.020761 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78341bb-8a1c-4f9c-8c7c-64f9bf3e1343 admin admin] REQUEST_METHOD: `PUT` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:44.021044 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78341bb-8a1c-4f9c-8c7c-64f9bf3e1343 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:44.021323 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78341bb-8a1c-4f9c-8c7c-64f9bf3e1343 admin admin] PATH_INFO: `/v3/projects/b9f83a8cb78d4c7face46c6d787bfc5d/users/69bb7d5c41bb4e46b1b0bea2f7bad45f/roles/cbf5c4c26898413d93e73b46b41046a3` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:44.035909 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78341bb-8a1c-4f9c-8c7c-64f9bf3e1343 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9f83a8cb78d4c7face46c6d787bfc5d, user_id=69bb7d5c41bb4e46b1b0bea2f7bad45f, role_id=cbf5c4c26898413d93e73b46b41046a3)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:44.038620 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78341bb-8a1c-4f9c-8c7c-64f9bf3e1343 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:44.076648 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-7812f8bf-8d72-4f8d-a17f-1b86193a8e2c admin admin] The token cache is being invalidate because project 9e7df6c98bc241398fb734d693186ca0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80469) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 18 05:17:44.077151 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-7812f8bf-8d72-4f8d-a17f-1b86193a8e2c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:44.077553 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-7812f8bf-8d72-4f8d-a17f-1b86193a8e2c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:44.079153 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1042/2057] 10.222.0.23 () {58 vars in 1232 bytes} [Sat Jun 18 05:17:43 2022] DELETE /identity/v3/projects/9e7df6c98bc241398fb734d693186ca0 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:17:44.084666 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bc0151-a5d9-4889-8422-4c00cc0ed454 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:44.084957 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bc0151-a5d9-4889-8422-4c00cc0ed454 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:44.085240 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bc0151-a5d9-4889-8422-4c00cc0ed454 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:44.104356 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-b9bc0151-a5d9-4889-8422-4c00cc0ed454 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:17:44.105654 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-b9bc0151-a5d9-4889-8422-4c00cc0ed454 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:17:44.113750 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1043/2058] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:17:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 18 05:17:44.122309 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebaa575-36f4-4328-ab95-b375d01ddcee None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:44.122594 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebaa575-36f4-4328-ab95-b375d01ddcee None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:44.122868 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebaa575-36f4-4328-ab95-b375d01ddcee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:44.134867 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1016/2059] 10.222.0.23 () {60 vars in 1482 bytes} [Sat Jun 18 05:17:43 2022] PUT /identity/v3/projects/b9f83a8cb78d4c7face46c6d787bfc5d/users/69bb7d5c41bb4e46b1b0bea2f7bad45f/roles/cbf5c4c26898413d93e73b46b41046a3 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 18 05:17:44.142564 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-2ebaa575-36f4-4328-ab95-b375d01ddcee None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:17:44.143838 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-2ebaa575-36f4-4328-ab95-b375d01ddcee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:17:44.145935 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f89a4a6-8020-41b5-abc8-f7a26e501a6b None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:44.146360 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f89a4a6-8020-41b5-abc8-f7a26e501a6b None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:44.146670 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f89a4a6-8020-41b5-abc8-f7a26e501a6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:44.184558 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-5f89a4a6-8020-41b5-abc8-f7a26e501a6b None None] MFA Rules not processed for user `69bb7d5c41bb4e46b1b0bea2f7bad45f`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:17:44.185844 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-5f89a4a6-8020-41b5-abc8-f7a26e501a6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:17:44.211894 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1044/2060] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:17:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 18 05:17:44.216364 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebaa575-36f4-4328-ab95-b375d01ddcee None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:44.225151 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebaa575-36f4-4328-ab95-b375d01ddcee None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:44.230432 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85494352-2740-4ca6-bb7f-f853eb80596a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:44.231083 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85494352-2740-4ca6-bb7f-f853eb80596a admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:44.231366 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85494352-2740-4ca6-bb7f-f853eb80596a admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:44.231641 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85494352-2740-4ca6-bb7f-f853eb80596a admin admin] PATH_INFO: `/v3/domains` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:44.232369 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85494352-2740-4ca6-bb7f-f853eb80596a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 05:17:44.232675 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85494352-2740-4ca6-bb7f-f853eb80596a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:44.235764 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85494352-2740-4ca6-bb7f-f853eb80596a admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:44.236303 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:17:44.248079 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1045/2061] 10.222.0.23 () {58 vars in 1152 bytes} [Sat Jun 18 05:17:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 18 05:17:44.251527 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85494352-2740-4ca6-bb7f-f853eb80596a admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:44.259280 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1017/2062] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:17:44 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 18 05:17:44.261888 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85494352-2740-4ca6-bb7f-f853eb80596a admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:44.268386 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b50847a-01da-4f4d-8b50-3afaf4d28698 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:44.269047 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b50847a-01da-4f4d-8b50-3afaf4d28698 admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:44.269416 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b50847a-01da-4f4d-8b50-3afaf4d28698 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:44.269786 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b50847a-01da-4f4d-8b50-3afaf4d28698 admin admin] PATH_INFO: `/v3/projects` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:44.270499 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b50847a-01da-4f4d-8b50-3afaf4d28698 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:44.271095 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f89a4a6-8020-41b5-abc8-f7a26e501a6b None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:44.272433 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b50847a-01da-4f4d-8b50-3afaf4d28698 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:44.273471 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-5f89a4a6-8020-41b5-abc8-f7a26e501a6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:17:44.332462 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f89a4a6-8020-41b5-abc8-f7a26e501a6b None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:44.336045 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ec5641-00f1-4d77-8342-962f11836c2c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9de0428abc8400aa338aba4f7c34c94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:44.336768 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ec5641-00f1-4d77-8342-962f11836c2c service cinder] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:44.337054 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ec5641-00f1-4d77-8342-962f11836c2c service cinder] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:44.337329 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ec5641-00f1-4d77-8342-962f11836c2c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:44.345665 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ec5641-00f1-4d77-8342-962f11836c2c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:44.347090 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ec5641-00f1-4d77-8342-962f11836c2c service cinder] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:44.356497 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1018/2063] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:17:44 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 18 05:17:44.358669 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1046/2064] 10.222.0.23 () {60 vars in 1152 bytes} [Sat Jun 18 05:17:44 2022] POST /identity/v3/projects => generated 375 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 18 05:17:44.362995 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ec5641-00f1-4d77-8342-962f11836c2c service cinder] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:44.374342 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ec5641-00f1-4d77-8342-962f11836c2c service cinder] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:44.439847 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bee208-c8ee-4812-8a15-9970f0e9ecc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:44.440512 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bee208-c8ee-4812-8a15-9970f0e9ecc4 admin admin] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:44.440797 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bee208-c8ee-4812-8a15-9970f0e9ecc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:44.441076 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bee208-c8ee-4812-8a15-9970f0e9ecc4 admin admin] PATH_INFO: `/v3/users` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:44.441808 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bee208-c8ee-4812-8a15-9970f0e9ecc4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:44.443726 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bee208-c8ee-4812-8a15-9970f0e9ecc4 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:44.557622 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1019/2065] 10.222.0.23 () {60 vars in 1143 bytes} [Sat Jun 18 05:17:44 2022] POST /identity/v3/users => generated 288 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:17:44.561670 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b50847a-01da-4f4d-8b50-3afaf4d28698 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:44.572986 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b50847a-01da-4f4d-8b50-3afaf4d28698 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:44.577634 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d6fe25-c22b-4560-9a15-c27b81ee0f02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:44.578283 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d6fe25-c22b-4560-9a15-c27b81ee0f02 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:44.578564 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d6fe25-c22b-4560-9a15-c27b81ee0f02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:44.578835 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d6fe25-c22b-4560-9a15-c27b81ee0f02 admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:44.579563 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d6fe25-c22b-4560-9a15-c27b81ee0f02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:44.581436 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d6fe25-c22b-4560-9a15-c27b81ee0f02 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:44.593374 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1047/2066] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:17:44 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:17:44.597055 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bee208-c8ee-4812-8a15-9970f0e9ecc4 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:44.608411 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bee208-c8ee-4812-8a15-9970f0e9ecc4 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:44.613139 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36140d66-e6de-430b-b119-fede794cacd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:44.613835 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36140d66-e6de-430b-b119-fede794cacd7 admin admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:44.614124 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36140d66-e6de-430b-b119-fede794cacd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:44.614404 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36140d66-e6de-430b-b119-fede794cacd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:44.615134 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36140d66-e6de-430b-b119-fede794cacd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:44.616731 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36140d66-e6de-430b-b119-fede794cacd7 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:44.630393 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1020/2067] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:17:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:17:44.632346 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d6fe25-c22b-4560-9a15-c27b81ee0f02 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:44.644967 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d6fe25-c22b-4560-9a15-c27b81ee0f02 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:44.651122 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11827e99-6ce7-449c-9cb2-ff2a555e4121 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:44.652237 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11827e99-6ce7-449c-9cb2-ff2a555e4121 admin admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:44.652526 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11827e99-6ce7-449c-9cb2-ff2a555e4121 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:44.652802 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11827e99-6ce7-449c-9cb2-ff2a555e4121 admin admin] PATH_INFO: `/v3/projects/74f98b8de4f24490a553848191f1b3c7/users/04a9356f01c841ecb369984ecce9bc0e/roles/cbf5c4c26898413d93e73b46b41046a3` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:44.682204 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11827e99-6ce7-449c-9cb2-ff2a555e4121 admin admin] RBAC: Authorizing `identity:create_grant(project_id=74f98b8de4f24490a553848191f1b3c7, user_id=04a9356f01c841ecb369984ecce9bc0e, role_id=cbf5c4c26898413d93e73b46b41046a3)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:44.682204 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11827e99-6ce7-449c-9cb2-ff2a555e4121 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:44.757119 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1048/2068] 10.222.0.23 () {60 vars in 1482 bytes} [Sat Jun 18 05:17:44 2022] PUT /identity/v3/projects/74f98b8de4f24490a553848191f1b3c7/users/04a9356f01c841ecb369984ecce9bc0e/roles/cbf5c4c26898413d93e73b46b41046a3 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:17:44.771930 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f343997f-d9fb-491f-8ecd-5d97d2e62190 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:44.772223 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f343997f-d9fb-491f-8ecd-5d97d2e62190 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:44.772505 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f343997f-d9fb-491f-8ecd-5d97d2e62190 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:44.803994 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-f343997f-d9fb-491f-8ecd-5d97d2e62190 None None] MFA Rules not processed for user `04a9356f01c841ecb369984ecce9bc0e`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:17:44.805256 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-f343997f-d9fb-491f-8ecd-5d97d2e62190 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:17:44.894977 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:17:44.900351 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1021/2069] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:17:44 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:17:44.923983 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11827e99-6ce7-449c-9cb2-ff2a555e4121 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:44.925775 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-11827e99-6ce7-449c-9cb2-ff2a555e4121 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:17:44.958877 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:17:44.986482 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11827e99-6ce7-449c-9cb2-ff2a555e4121 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:44.990626 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a32740e-cf79-4aa1-8ee8-6cf72d401ed7 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4638e5d1fa124667b07054a87c2d353e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:44.991468 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a32740e-cf79-4aa1-8ee8-6cf72d401ed7 service glance] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:44.991762 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a32740e-cf79-4aa1-8ee8-6cf72d401ed7 service glance] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:44.992037 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a32740e-cf79-4aa1-8ee8-6cf72d401ed7 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:45.001500 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a32740e-cf79-4aa1-8ee8-6cf72d401ed7 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:45.002728 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a32740e-cf79-4aa1-8ee8-6cf72d401ed7 service glance] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:45.014805 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1049/2070] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:17:44 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:17:47.139601 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f343997f-d9fb-491f-8ecd-5d97d2e62190 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:47.141106 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-f343997f-d9fb-491f-8ecd-5d97d2e62190 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:17:47.212727 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f343997f-d9fb-491f-8ecd-5d97d2e62190 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:47.218034 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ca7d87-c443-4e2b-a9f3-936b0c611115 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:47.218913 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ca7d87-c443-4e2b-a9f3-936b0c611115 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:47.219289 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ca7d87-c443-4e2b-a9f3-936b0c611115 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:47.219570 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ca7d87-c443-4e2b-a9f3-936b0c611115 admin admin] PATH_INFO: `/v3/users/b17031a1b654414a9737632fa471c1b8` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:47.220798 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ca7d87-c443-4e2b-a9f3-936b0c611115 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b17031a1b654414a9737632fa471c1b8)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:47.222835 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ca7d87-c443-4e2b-a9f3-936b0c611115 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:47.352539 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-03ca7d87-c443-4e2b-a9f3-936b0c611115 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b17031a1b654414a9737632fa471c1b8'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:47.353968 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-03ca7d87-c443-4e2b-a9f3-936b0c611115 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b17031a1b654414a9737632fa471c1b8'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:47.396210 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-03ca7d87-c443-4e2b-a9f3-936b0c611115 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b17031a1b654414a9737632fa471c1b8'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:47.410089 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-03ca7d87-c443-4e2b-a9f3-936b0c611115 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b17031a1b654414a9737632fa471c1b8'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:47.417916 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-03ca7d87-c443-4e2b-a9f3-936b0c611115 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b17031a1b654414a9737632fa471c1b8'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:47.425374 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1022/2071] 10.222.0.23 () {58 vars in 1223 bytes} [Sat Jun 18 05:17:47 2022] DELETE /identity/v3/users/b17031a1b654414a9737632fa471c1b8 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 18 05:17:47.437438 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaa601d-d261-4999-a890-2a9f280c0869 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:47.437438 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaa601d-d261-4999-a890-2a9f280c0869 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:47.437438 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaa601d-d261-4999-a890-2a9f280c0869 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:47.455155 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-abaa601d-d261-4999-a890-2a9f280c0869 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:17:47.456399 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-abaa601d-d261-4999-a890-2a9f280c0869 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:17:47.517433 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1050/2072] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:17:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:17:47.527522 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ca7d87-c443-4e2b-a9f3-936b0c611115 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:47.529204 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-03ca7d87-c443-4e2b-a9f3-936b0c611115 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:17:47.591790 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ca7d87-c443-4e2b-a9f3-936b0c611115 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:47.595500 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87939486-30cf-4f8d-ab78-9ab52fd1a738 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c4d7cde8f4ad9aa479bb09e325db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:47.596149 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87939486-30cf-4f8d-ab78-9ab52fd1a738 service neutron] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:47.596435 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87939486-30cf-4f8d-ab78-9ab52fd1a738 service neutron] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:47.596761 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87939486-30cf-4f8d-ab78-9ab52fd1a738 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:47.605912 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87939486-30cf-4f8d-ab78-9ab52fd1a738 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:47.607347 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87939486-30cf-4f8d-ab78-9ab52fd1a738 service neutron] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:47.617288 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1023/2073] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:17:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:17:48.016763 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaa601d-d261-4999-a890-2a9f280c0869 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:48.018334 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-abaa601d-d261-4999-a890-2a9f280c0869 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:17:48.033304 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaa601d-d261-4999-a890-2a9f280c0869 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:48.038960 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0404e0-ad7a-4026-b1a9-f5336c5d7277 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:48.041047 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0404e0-ad7a-4026-b1a9-f5336c5d7277 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:48.041659 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0404e0-ad7a-4026-b1a9-f5336c5d7277 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:48.041933 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0404e0-ad7a-4026-b1a9-f5336c5d7277 admin admin] PATH_INFO: `/v3/projects/8fa0c1bb956f4efbac54fdfb60f6573c` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:48.043339 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0404e0-ad7a-4026-b1a9-f5336c5d7277 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8fa0c1bb956f4efbac54fdfb60f6573c)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:48.045762 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0404e0-ad7a-4026-b1a9-f5336c5d7277 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:48.164832 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-5f0404e0-ad7a-4026-b1a9-f5336c5d7277 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8fa0c1bb956f4efbac54fdfb60f6573c', 'request_id': 'req-5f0404e0-ad7a-4026-b1a9-f5336c5d7277'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:48.171043 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-5f0404e0-ad7a-4026-b1a9-f5336c5d7277 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8fa0c1bb956f4efbac54fdfb60f6573c', 'request_id': 'req-5f0404e0-ad7a-4026-b1a9-f5336c5d7277'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:48.178191 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-5f0404e0-ad7a-4026-b1a9-f5336c5d7277 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8fa0c1bb956f4efbac54fdfb60f6573c', 'request_id': 'req-5f0404e0-ad7a-4026-b1a9-f5336c5d7277'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:48.287519 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-5f0404e0-ad7a-4026-b1a9-f5336c5d7277 admin admin] The token cache is being invalidate because project 8fa0c1bb956f4efbac54fdfb60f6573c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80469) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 18 05:17:48.287809 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-5f0404e0-ad7a-4026-b1a9-f5336c5d7277 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:48.288473 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-5f0404e0-ad7a-4026-b1a9-f5336c5d7277 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:48.290093 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1051/2074] 10.222.0.23 () {58 vars in 1232 bytes} [Sat Jun 18 05:17:48 2022] DELETE /identity/v3/projects/8fa0c1bb956f4efbac54fdfb60f6573c => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:17:48.295112 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82578f3f-22d6-41e8-a7e6-d54744b7e6c7 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:48.295611 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82578f3f-22d6-41e8-a7e6-d54744b7e6c7 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:48.295720 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82578f3f-22d6-41e8-a7e6-d54744b7e6c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:48.313426 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-82578f3f-22d6-41e8-a7e6-d54744b7e6c7 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:17:48.314679 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-82578f3f-22d6-41e8-a7e6-d54744b7e6c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:17:48.332659 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:17:48.378763 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1024/2075] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:17:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 18 05:17:48.387166 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577e96fc-0227-41dd-9bac-ea54ee3c1c95 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:48.387577 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577e96fc-0227-41dd-9bac-ea54ee3c1c95 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:48.387936 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577e96fc-0227-41dd-9bac-ea54ee3c1c95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:48.408112 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-577e96fc-0227-41dd-9bac-ea54ee3c1c95 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:17:48.409443 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-577e96fc-0227-41dd-9bac-ea54ee3c1c95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:17:48.418486 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1052/2076] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:17:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:17:48.422616 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82578f3f-22d6-41e8-a7e6-d54744b7e6c7 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:48.437216 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82578f3f-22d6-41e8-a7e6-d54744b7e6c7 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:48.442275 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b5aac2-31f6-4788-a9b7-b71c72372e51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:48.443081 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b5aac2-31f6-4788-a9b7-b71c72372e51 admin admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:48.443471 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b5aac2-31f6-4788-a9b7-b71c72372e51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:48.443852 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b5aac2-31f6-4788-a9b7-b71c72372e51 admin admin] PATH_INFO: `/v3/domains` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:48.444693 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b5aac2-31f6-4788-a9b7-b71c72372e51 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 05:17:48.445286 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b5aac2-31f6-4788-a9b7-b71c72372e51 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:48.448873 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b5aac2-31f6-4788-a9b7-b71c72372e51 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:48.464995 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1025/2077] 10.222.0.23 () {58 vars in 1152 bytes} [Sat Jun 18 05:17:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 05:17:48.466135 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577e96fc-0227-41dd-9bac-ea54ee3c1c95 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:48.477064 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577e96fc-0227-41dd-9bac-ea54ee3c1c95 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:48.494458 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474113af-9c38-424b-ace8-ed50a9c1589a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:48.495152 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474113af-9c38-424b-ace8-ed50a9c1589a admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:48.495442 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474113af-9c38-424b-ace8-ed50a9c1589a admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:48.495723 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474113af-9c38-424b-ace8-ed50a9c1589a admin admin] PATH_INFO: `/v3/projects` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:48.496451 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474113af-9c38-424b-ace8-ed50a9c1589a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:48.498587 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474113af-9c38-424b-ace8-ed50a9c1589a admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:48.595220 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1053/2078] 10.222.0.23 () {60 vars in 1152 bytes} [Sat Jun 18 05:17:48 2022] POST /identity/v3/projects => generated 385 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 18 05:17:48.599286 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b5aac2-31f6-4788-a9b7-b71c72372e51 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:48.610903 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b5aac2-31f6-4788-a9b7-b71c72372e51 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:48.679717 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a133a271-b1e8-4bc2-addb-30bd2c18ab58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:48.680504 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a133a271-b1e8-4bc2-addb-30bd2c18ab58 admin admin] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:48.681412 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a133a271-b1e8-4bc2-addb-30bd2c18ab58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:48.681974 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a133a271-b1e8-4bc2-addb-30bd2c18ab58 admin admin] PATH_INFO: `/v3/users` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:48.683101 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a133a271-b1e8-4bc2-addb-30bd2c18ab58 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:48.685537 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a133a271-b1e8-4bc2-addb-30bd2c18ab58 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:48.788751 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1026/2079] 10.222.0.23 () {60 vars in 1143 bytes} [Sat Jun 18 05:17:48 2022] POST /identity/v3/users => generated 293 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:17:48.791936 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474113af-9c38-424b-ace8-ed50a9c1589a admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:48.804523 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474113af-9c38-424b-ace8-ed50a9c1589a admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:48.810318 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be480199-46f9-4518-9547-b784a2fdb120 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:48.810977 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be480199-46f9-4518-9547-b784a2fdb120 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:48.811258 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be480199-46f9-4518-9547-b784a2fdb120 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:48.811726 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be480199-46f9-4518-9547-b784a2fdb120 admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:48.812453 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be480199-46f9-4518-9547-b784a2fdb120 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:48.814099 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be480199-46f9-4518-9547-b784a2fdb120 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:48.831271 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1054/2080] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:17:48 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:17:48.832653 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a133a271-b1e8-4bc2-addb-30bd2c18ab58 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:48.844182 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a133a271-b1e8-4bc2-addb-30bd2c18ab58 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:48.849301 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a98ddf-0340-4f55-aba5-ed0013b981af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:48.849995 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a98ddf-0340-4f55-aba5-ed0013b981af admin admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:48.850390 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a98ddf-0340-4f55-aba5-ed0013b981af admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:48.850668 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a98ddf-0340-4f55-aba5-ed0013b981af admin admin] PATH_INFO: `/v3/roles` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:48.851401 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a98ddf-0340-4f55-aba5-ed0013b981af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:48.854639 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a98ddf-0340-4f55-aba5-ed0013b981af admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:48.868122 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1027/2081] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:17:48 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:17:48.869996 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be480199-46f9-4518-9547-b784a2fdb120 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:48.880548 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be480199-46f9-4518-9547-b784a2fdb120 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:48.885586 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e60336-8fc2-4b3d-a4df-781ecd07da96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:48.886471 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e60336-8fc2-4b3d-a4df-781ecd07da96 admin admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:48.886751 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e60336-8fc2-4b3d-a4df-781ecd07da96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:48.887052 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e60336-8fc2-4b3d-a4df-781ecd07da96 admin admin] PATH_INFO: `/v3/projects/3a833f00ba294564a66b9eefbc49147e/users/113fba3872764b02bcbcc69299d8768f/roles/cbf5c4c26898413d93e73b46b41046a3` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:48.899244 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:17:48.903851 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e60336-8fc2-4b3d-a4df-781ecd07da96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a833f00ba294564a66b9eefbc49147e, user_id=113fba3872764b02bcbcc69299d8768f, role_id=cbf5c4c26898413d93e73b46b41046a3)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:48.906853 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e60336-8fc2-4b3d-a4df-781ecd07da96 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:48.991810 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1055/2082] 10.222.0.23 () {60 vars in 1482 bytes} [Sat Jun 18 05:17:48 2022] PUT /identity/v3/projects/3a833f00ba294564a66b9eefbc49147e/users/113fba3872764b02bcbcc69299d8768f/roles/cbf5c4c26898413d93e73b46b41046a3 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:17:48.998130 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59414748-d3f4-460d-bf8b-fdcb5dd6f8fa None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:48.998421 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59414748-d3f4-460d-bf8b-fdcb5dd6f8fa None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:48.998723 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59414748-d3f4-460d-bf8b-fdcb5dd6f8fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:49.019450 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-59414748-d3f4-460d-bf8b-fdcb5dd6f8fa None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:17:49.021100 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-59414748-d3f4-460d-bf8b-fdcb5dd6f8fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:17:49.089001 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1028/2083] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:17:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:17:49.105417 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e60336-8fc2-4b3d-a4df-781ecd07da96 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:49.106884 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-96e60336-8fc2-4b3d-a4df-781ecd07da96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:17:49.201840 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e60336-8fc2-4b3d-a4df-781ecd07da96 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:49.206344 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851443e5-3898-4cb7-95a3-1cc51f45aea4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c4d7cde8f4ad9aa479bb09e325db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:49.206519 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851443e5-3898-4cb7-95a3-1cc51f45aea4 service neutron] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:49.209501 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851443e5-3898-4cb7-95a3-1cc51f45aea4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:49.209501 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851443e5-3898-4cb7-95a3-1cc51f45aea4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:49.223466 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851443e5-3898-4cb7-95a3-1cc51f45aea4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:49.224997 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851443e5-3898-4cb7-95a3-1cc51f45aea4 service neutron] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:49.237405 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1056/2084] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:17:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:17:51.664153 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59414748-d3f4-460d-bf8b-fdcb5dd6f8fa None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:51.667317 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-59414748-d3f4-460d-bf8b-fdcb5dd6f8fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:17:51.691137 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59414748-d3f4-460d-bf8b-fdcb5dd6f8fa None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:51.701521 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45824b3-55c0-485b-928d-656b84bb0e92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:51.702647 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45824b3-55c0-485b-928d-656b84bb0e92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:51.702935 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45824b3-55c0-485b-928d-656b84bb0e92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:51.703266 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45824b3-55c0-485b-928d-656b84bb0e92 admin admin] PATH_INFO: `/v3/users/79f8817a70b844d7ac49543d74d0e38b` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:51.705118 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45824b3-55c0-485b-928d-656b84bb0e92 admin admin] RBAC: Authorizing `identity:delete_user(user_id=79f8817a70b844d7ac49543d74d0e38b)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:51.707581 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45824b3-55c0-485b-928d-656b84bb0e92 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:51.736319 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:17:51.974522 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-a45824b3-55c0-485b-928d-656b84bb0e92 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79f8817a70b844d7ac49543d74d0e38b'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:51.978661 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-a45824b3-55c0-485b-928d-656b84bb0e92 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79f8817a70b844d7ac49543d74d0e38b'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:52.048093 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-a45824b3-55c0-485b-928d-656b84bb0e92 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79f8817a70b844d7ac49543d74d0e38b'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:52.069351 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-a45824b3-55c0-485b-928d-656b84bb0e92 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79f8817a70b844d7ac49543d74d0e38b'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:52.082449 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-a45824b3-55c0-485b-928d-656b84bb0e92 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79f8817a70b844d7ac49543d74d0e38b'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:52.090401 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1029/2085] 10.222.0.23 () {58 vars in 1223 bytes} [Sat Jun 18 05:17:51 2022] DELETE /identity/v3/users/79f8817a70b844d7ac49543d74d0e38b => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:17:52.755702 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851443e5-3898-4cb7-95a3-1cc51f45aea4 service neutron] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:52.772913 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851443e5-3898-4cb7-95a3-1cc51f45aea4 service neutron] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:52.940454 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4b8611-d51a-4ba8-b104-9ce7057d02f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:52.941667 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4b8611-d51a-4ba8-b104-9ce7057d02f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:52.942016 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4b8611-d51a-4ba8-b104-9ce7057d02f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:52.942281 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4b8611-d51a-4ba8-b104-9ce7057d02f6 admin admin] PATH_INFO: `/v3/projects/0b73d42a085a45bfb3ccced90b190a5b` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:52.943567 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4b8611-d51a-4ba8-b104-9ce7057d02f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b73d42a085a45bfb3ccced90b190a5b)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:52.945502 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4b8611-d51a-4ba8-b104-9ce7057d02f6 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:53.091111 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-2c4b8611-d51a-4ba8-b104-9ce7057d02f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b73d42a085a45bfb3ccced90b190a5b', 'request_id': 'req-2c4b8611-d51a-4ba8-b104-9ce7057d02f6'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:53.098491 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-2c4b8611-d51a-4ba8-b104-9ce7057d02f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b73d42a085a45bfb3ccced90b190a5b', 'request_id': 'req-2c4b8611-d51a-4ba8-b104-9ce7057d02f6'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:53.106042 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-2c4b8611-d51a-4ba8-b104-9ce7057d02f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b73d42a085a45bfb3ccced90b190a5b', 'request_id': 'req-2c4b8611-d51a-4ba8-b104-9ce7057d02f6'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:53.178820 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-2c4b8611-d51a-4ba8-b104-9ce7057d02f6 admin admin] The token cache is being invalidate because project 0b73d42a085a45bfb3ccced90b190a5b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80469) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 18 05:17:53.179190 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-2c4b8611-d51a-4ba8-b104-9ce7057d02f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:53.179822 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-2c4b8611-d51a-4ba8-b104-9ce7057d02f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:17:53.182039 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1057/2086] 10.222.0.23 () {58 vars in 1232 bytes} [Sat Jun 18 05:17:52 2022] DELETE /identity/v3/projects/0b73d42a085a45bfb3ccced90b190a5b => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:17:53.187259 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b92a2e4-8d17-4547-b20d-7ee63321e483 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:53.187728 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b92a2e4-8d17-4547-b20d-7ee63321e483 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:53.187991 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b92a2e4-8d17-4547-b20d-7ee63321e483 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:53.208078 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-2b92a2e4-8d17-4547-b20d-7ee63321e483 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:17:53.209536 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-2b92a2e4-8d17-4547-b20d-7ee63321e483 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:17:53.284610 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1030/2087] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:17:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:17:53.292823 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314edf39-b393-4663-9111-56491e1337d1 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:53.293177 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314edf39-b393-4663-9111-56491e1337d1 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:53.294087 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314edf39-b393-4663-9111-56491e1337d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:53.336790 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-314edf39-b393-4663-9111-56491e1337d1 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:17:53.338638 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-314edf39-b393-4663-9111-56491e1337d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:17:53.348808 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1058/2088] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:17:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:17:53.354077 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b92a2e4-8d17-4547-b20d-7ee63321e483 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:53.382424 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b92a2e4-8d17-4547-b20d-7ee63321e483 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:53.392819 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d49dcd9-bd37-4cb7-b440-e24c6bd82f53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:53.394371 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d49dcd9-bd37-4cb7-b440-e24c6bd82f53 admin admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:53.394786 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d49dcd9-bd37-4cb7-b440-e24c6bd82f53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:53.395163 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d49dcd9-bd37-4cb7-b440-e24c6bd82f53 admin admin] PATH_INFO: `/v3/domains` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:53.397008 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d49dcd9-bd37-4cb7-b440-e24c6bd82f53 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 05:17:53.397008 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d49dcd9-bd37-4cb7-b440-e24c6bd82f53 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:53.400204 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d49dcd9-bd37-4cb7-b440-e24c6bd82f53 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:53.418989 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1031/2089] 10.222.0.23 () {58 vars in 1152 bytes} [Sat Jun 18 05:17:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 05:17:53.423286 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314edf39-b393-4663-9111-56491e1337d1 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:53.437044 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314edf39-b393-4663-9111-56491e1337d1 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:53.449939 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29dbdaf-d00f-420b-b6a3-dd2b45101855 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:53.455174 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29dbdaf-d00f-420b-b6a3-dd2b45101855 admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:53.455174 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29dbdaf-d00f-420b-b6a3-dd2b45101855 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:53.455174 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29dbdaf-d00f-420b-b6a3-dd2b45101855 admin admin] PATH_INFO: `/v3/projects` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:53.455174 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29dbdaf-d00f-420b-b6a3-dd2b45101855 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:53.455459 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29dbdaf-d00f-420b-b6a3-dd2b45101855 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:53.470678 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:17:53.571362 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1059/2090] 10.222.0.23 () {60 vars in 1152 bytes} [Sat Jun 18 05:17:53 2022] POST /identity/v3/projects => generated 393 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:17:53.571683 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d49dcd9-bd37-4cb7-b440-e24c6bd82f53 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:53.583078 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d49dcd9-bd37-4cb7-b440-e24c6bd82f53 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:53.661294 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630c1e8a-6bd8-4304-adf1-a35338160939 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:53.662382 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630c1e8a-6bd8-4304-adf1-a35338160939 admin admin] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:53.662382 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630c1e8a-6bd8-4304-adf1-a35338160939 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:53.663140 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630c1e8a-6bd8-4304-adf1-a35338160939 admin admin] PATH_INFO: `/v3/users` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:53.663855 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630c1e8a-6bd8-4304-adf1-a35338160939 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:53.666955 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630c1e8a-6bd8-4304-adf1-a35338160939 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:53.788666 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1032/2091] 10.222.0.23 () {60 vars in 1143 bytes} [Sat Jun 18 05:17:53 2022] POST /identity/v3/users => generated 297 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 18 05:17:53.793476 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29dbdaf-d00f-420b-b6a3-dd2b45101855 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:53.804326 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29dbdaf-d00f-420b-b6a3-dd2b45101855 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:53.811413 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb8d5f5-729b-446b-a4d3-5480b82f05a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:53.812101 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb8d5f5-729b-446b-a4d3-5480b82f05a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:53.812431 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb8d5f5-729b-446b-a4d3-5480b82f05a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:53.812711 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb8d5f5-729b-446b-a4d3-5480b82f05a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:53.813483 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb8d5f5-729b-446b-a4d3-5480b82f05a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:53.815309 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb8d5f5-729b-446b-a4d3-5480b82f05a0 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:53.849938 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1060/2092] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:17:53 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:17:53.862697 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630c1e8a-6bd8-4304-adf1-a35338160939 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:53.888216 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630c1e8a-6bd8-4304-adf1-a35338160939 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:53.895934 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da65fc9-ca26-4e89-b9b3-d71160935c2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:53.896605 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da65fc9-ca26-4e89-b9b3-d71160935c2d admin admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:53.896897 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da65fc9-ca26-4e89-b9b3-d71160935c2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:53.897173 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da65fc9-ca26-4e89-b9b3-d71160935c2d admin admin] PATH_INFO: `/v3/roles` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:53.898057 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da65fc9-ca26-4e89-b9b3-d71160935c2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:53.899739 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da65fc9-ca26-4e89-b9b3-d71160935c2d admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:53.917468 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1033/2093] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:17:53 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:17:53.922441 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb8d5f5-729b-446b-a4d3-5480b82f05a0 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:53.932984 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb8d5f5-729b-446b-a4d3-5480b82f05a0 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:53.938271 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3581dda-1f7f-46ab-bb6a-d63ea5f1e777 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:53.939078 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3581dda-1f7f-46ab-bb6a-d63ea5f1e777 admin admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:53.939482 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3581dda-1f7f-46ab-bb6a-d63ea5f1e777 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:53.939811 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3581dda-1f7f-46ab-bb6a-d63ea5f1e777 admin admin] PATH_INFO: `/v3/projects/8820096253064e5abbe50f23bdc3ebac/users/216d2b492c55410d9563051142048160/roles/cbf5c4c26898413d93e73b46b41046a3` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:53.955994 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3581dda-1f7f-46ab-bb6a-d63ea5f1e777 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8820096253064e5abbe50f23bdc3ebac, user_id=216d2b492c55410d9563051142048160, role_id=cbf5c4c26898413d93e73b46b41046a3)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:53.958707 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3581dda-1f7f-46ab-bb6a-d63ea5f1e777 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:54.037344 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1061/2094] 10.222.0.23 () {60 vars in 1482 bytes} [Sat Jun 18 05:17:53 2022] PUT /identity/v3/projects/8820096253064e5abbe50f23bdc3ebac/users/216d2b492c55410d9563051142048160/roles/cbf5c4c26898413d93e73b46b41046a3 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:17:54.038880 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffe703a-dfcf-4504-b3ca-4c602fa37811 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:54.039268 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffe703a-dfcf-4504-b3ca-4c602fa37811 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:54.039618 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffe703a-dfcf-4504-b3ca-4c602fa37811 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:54.052826 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:17:54.059391 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-fffe703a-dfcf-4504-b3ca-4c602fa37811 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:17:54.060796 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-fffe703a-dfcf-4504-b3ca-4c602fa37811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:17:54.124554 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1034/2095] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:17:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:17:54.134791 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3581dda-1f7f-46ab-bb6a-d63ea5f1e777 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:54.136798 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-c3581dda-1f7f-46ab-bb6a-d63ea5f1e777 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:17:54.215883 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3581dda-1f7f-46ab-bb6a-d63ea5f1e777 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:54.222446 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83777643-26b8-40f9-a61a-daf4772bf165 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c4d7cde8f4ad9aa479bb09e325db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:54.223131 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83777643-26b8-40f9-a61a-daf4772bf165 service neutron] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:54.223544 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83777643-26b8-40f9-a61a-daf4772bf165 service neutron] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:54.224665 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83777643-26b8-40f9-a61a-daf4772bf165 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:54.246101 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83777643-26b8-40f9-a61a-daf4772bf165 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:54.262037 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83777643-26b8-40f9-a61a-daf4772bf165 service neutron] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:54.276901 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1062/2096] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:17:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 18 05:17:55.421610 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38b0661-9185-4643-9887-d364f6140635 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:55.422168 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38b0661-9185-4643-9887-d364f6140635 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:55.422451 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38b0661-9185-4643-9887-d364f6140635 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:55.491239 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-e38b0661-9185-4643-9887-d364f6140635 None None] MFA Rules not processed for user `113fba3872764b02bcbcc69299d8768f`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:17:55.492754 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-e38b0661-9185-4643-9887-d364f6140635 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:17:55.609847 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1035/2097] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:17:55 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:17:55.614088 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83777643-26b8-40f9-a61a-daf4772bf165 service neutron] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:55.616182 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-83777643-26b8-40f9-a61a-daf4772bf165 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:17:55.631567 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83777643-26b8-40f9-a61a-daf4772bf165 service neutron] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:55.635873 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5b4ef5-acbb-4596-b7d3-8dd55fbf39a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:55.636552 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5b4ef5-acbb-4596-b7d3-8dd55fbf39a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:55.636837 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5b4ef5-acbb-4596-b7d3-8dd55fbf39a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:55.637111 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5b4ef5-acbb-4596-b7d3-8dd55fbf39a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:55.637997 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5b4ef5-acbb-4596-b7d3-8dd55fbf39a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:55.639942 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5b4ef5-acbb-4596-b7d3-8dd55fbf39a8 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:55.758221 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1063/2098] 10.222.0.23 () {60 vars in 1152 bytes} [Sat Jun 18 05:17:55 2022] POST /identity/v3/projects => generated 383 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:17:55.762706 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38b0661-9185-4643-9887-d364f6140635 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:55.774112 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38b0661-9185-4643-9887-d364f6140635 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:55.853602 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429142c0-0c12-4837-a609-b72ca7b18107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:55.854332 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429142c0-0c12-4837-a609-b72ca7b18107 admin admin] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:55.854657 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429142c0-0c12-4837-a609-b72ca7b18107 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:55.855083 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429142c0-0c12-4837-a609-b72ca7b18107 admin admin] PATH_INFO: `/v3/users` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:55.855782 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429142c0-0c12-4837-a609-b72ca7b18107 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:55.857887 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429142c0-0c12-4837-a609-b72ca7b18107 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:55.952840 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1036/2099] 10.222.0.23 () {60 vars in 1143 bytes} [Sat Jun 18 05:17:55 2022] POST /identity/v3/users => generated 298 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:17:55.956402 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5b4ef5-acbb-4596-b7d3-8dd55fbf39a8 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:55.981006 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5b4ef5-acbb-4596-b7d3-8dd55fbf39a8 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:55.987044 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0dab9d-1d42-4abb-bc6b-0b11413a936f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:55.987731 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0dab9d-1d42-4abb-bc6b-0b11413a936f admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:55.988018 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0dab9d-1d42-4abb-bc6b-0b11413a936f admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:55.988493 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0dab9d-1d42-4abb-bc6b-0b11413a936f admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:55.989734 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0dab9d-1d42-4abb-bc6b-0b11413a936f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:55.992123 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0dab9d-1d42-4abb-bc6b-0b11413a936f admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:56.010790 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1064/2100] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:17:55 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 18 05:17:56.012098 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429142c0-0c12-4837-a609-b72ca7b18107 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:56.023475 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429142c0-0c12-4837-a609-b72ca7b18107 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:56.031804 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c976bc65-8a74-41c6-a93c-57efcc210be1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:56.032740 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c976bc65-8a74-41c6-a93c-57efcc210be1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:56.033029 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c976bc65-8a74-41c6-a93c-57efcc210be1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:56.033309 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c976bc65-8a74-41c6-a93c-57efcc210be1 admin admin] PATH_INFO: `/v3/projects/fcb1e990c65747549be861c1103aafb1/users/2d63e07595cb4ebcafb24f6928f06f61/roles/21c65f2c51bd4dd283d13507d2714467` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:56.053605 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c976bc65-8a74-41c6-a93c-57efcc210be1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fcb1e990c65747549be861c1103aafb1, user_id=2d63e07595cb4ebcafb24f6928f06f61, role_id=21c65f2c51bd4dd283d13507d2714467)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:56.056486 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c976bc65-8a74-41c6-a93c-57efcc210be1 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:56.145869 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1037/2101] 10.222.0.23 () {60 vars in 1482 bytes} [Sat Jun 18 05:17:56 2022] PUT /identity/v3/projects/fcb1e990c65747549be861c1103aafb1/users/2d63e07595cb4ebcafb24f6928f06f61/roles/21c65f2c51bd4dd283d13507d2714467 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:17:56.148451 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0dab9d-1d42-4abb-bc6b-0b11413a936f admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:56.166776 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0dab9d-1d42-4abb-bc6b-0b11413a936f admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:56.291286 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d9d4a9-5b06-4316-8d2c-e6ab69f9e963 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:56.292084 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d9d4a9-5b06-4316-8d2c-e6ab69f9e963 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:56.292442 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d9d4a9-5b06-4316-8d2c-e6ab69f9e963 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:56.292812 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d9d4a9-5b06-4316-8d2c-e6ab69f9e963 admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:56.293668 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d9d4a9-5b06-4316-8d2c-e6ab69f9e963 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:56.296029 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d9d4a9-5b06-4316-8d2c-e6ab69f9e963 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:56.316644 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1065/2102] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:17:56 2022] GET /identity/v3/roles => generated 1420 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:17:56.319127 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c976bc65-8a74-41c6-a93c-57efcc210be1 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:17:56.326950 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:17:56.341114 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c976bc65-8a74-41c6-a93c-57efcc210be1 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:17:56.350107 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efa26e0-ec7f-47f7-b991-ec690e625b64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:17:56.351165 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efa26e0-ec7f-47f7-b991-ec690e625b64 admin admin] REQUEST_METHOD: `PUT` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:17:56.351550 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efa26e0-ec7f-47f7-b991-ec690e625b64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:17:56.351931 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efa26e0-ec7f-47f7-b991-ec690e625b64 admin admin] PATH_INFO: `/v3/projects/fcb1e990c65747549be861c1103aafb1/users/2d63e07595cb4ebcafb24f6928f06f61/roles/21c65f2c51bd4dd283d13507d2714467` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:17:56.354738 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efa26e0-ec7f-47f7-b991-ec690e625b64 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fcb1e990c65747549be861c1103aafb1, user_id=2d63e07595cb4ebcafb24f6928f06f61, role_id=21c65f2c51bd4dd283d13507d2714467)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:17:56.357737 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efa26e0-ec7f-47f7-b991-ec690e625b64 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:17:56.373986 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1038/2103] 10.222.0.23 () {60 vars in 1482 bytes} [Sat Jun 18 05:17:56 2022] PUT /identity/v3/projects/fcb1e990c65747549be861c1103aafb1/users/2d63e07595cb4ebcafb24f6928f06f61/roles/21c65f2c51bd4dd283d13507d2714467 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:18:01.702739 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56c194f-a840-41c9-9844-ffd8ff7e684c None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:18:01.703388 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56c194f-a840-41c9-9844-ffd8ff7e684c None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:18:01.703388 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56c194f-a840-41c9-9844-ffd8ff7e684c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:18:01.742519 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-c56c194f-a840-41c9-9844-ffd8ff7e684c None None] MFA Rules not processed for user `216d2b492c55410d9563051142048160`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:18:01.745187 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-c56c194f-a840-41c9-9844-ffd8ff7e684c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:18:01.765510 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:18:01.841192 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1066/2104] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:18:01 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:18:01.852260 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e9c78d-95b0-413f-99d1-d2a61f16be1a None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:18:01.852929 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e9c78d-95b0-413f-99d1-d2a61f16be1a None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:18:01.852929 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e9c78d-95b0-413f-99d1-d2a61f16be1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:18:01.876430 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-40e9c78d-95b0-413f-99d1-d2a61f16be1a None None] MFA Rules not processed for user `216d2b492c55410d9563051142048160`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:18:01.877846 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-40e9c78d-95b0-413f-99d1-d2a61f16be1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:18:01.896212 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1039/2105] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:18:01 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:18:01.913707 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56c194f-a840-41c9-9844-ffd8ff7e684c None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:18:01.915502 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-c56c194f-a840-41c9-9844-ffd8ff7e684c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:18:02.078464 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56c194f-a840-41c9-9844-ffd8ff7e684c None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:18:02.103596 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953da690-8ef2-48d9-b9fd-63504a9863bc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15e7f45ee0744c1d9602187eac8e8b80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:18:02.104348 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953da690-8ef2-48d9-b9fd-63504a9863bc service nova] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:18:02.105755 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953da690-8ef2-48d9-b9fd-63504a9863bc service nova] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:18:02.108553 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953da690-8ef2-48d9-b9fd-63504a9863bc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:18:02.130787 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953da690-8ef2-48d9-b9fd-63504a9863bc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:18:02.135413 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953da690-8ef2-48d9-b9fd-63504a9863bc service nova] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:18:02.190114 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1067/2106] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:18:01 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:18:03.295873 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f190ea-b9de-43a3-bc36-3dbde74fd59c None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:18:03.296380 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f190ea-b9de-43a3-bc36-3dbde74fd59c None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:18:03.296489 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f190ea-b9de-43a3-bc36-3dbde74fd59c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:18:03.333905 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-b4f190ea-b9de-43a3-bc36-3dbde74fd59c None None] MFA Rules not processed for user `216d2b492c55410d9563051142048160`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:18:03.335392 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-b4f190ea-b9de-43a3-bc36-3dbde74fd59c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:18:03.351217 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1040/2107] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:18:03 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:18:03.536996 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953da690-8ef2-48d9-b9fd-63504a9863bc service nova] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:18:03.539489 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-953da690-8ef2-48d9-b9fd-63504a9863bc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:18:03.748717 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953da690-8ef2-48d9-b9fd-63504a9863bc service nova] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:18:03.777807 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741b5c53-bab2-4654-bacf-4eed9dc8bae0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:18:03.779243 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741b5c53-bab2-4654-bacf-4eed9dc8bae0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:18:03.779761 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741b5c53-bab2-4654-bacf-4eed9dc8bae0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:18:03.780255 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741b5c53-bab2-4654-bacf-4eed9dc8bae0 admin admin] PATH_INFO: `/v3/users/9cda29438b944702b2dad445fa604697` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:18:03.782227 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741b5c53-bab2-4654-bacf-4eed9dc8bae0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9cda29438b944702b2dad445fa604697)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:18:03.784914 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741b5c53-bab2-4654-bacf-4eed9dc8bae0 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:18:04.009813 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:18:04.028589 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-741b5c53-bab2-4654-bacf-4eed9dc8bae0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9cda29438b944702b2dad445fa604697'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:18:04.072280 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-741b5c53-bab2-4654-bacf-4eed9dc8bae0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9cda29438b944702b2dad445fa604697'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:18:04.082314 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-741b5c53-bab2-4654-bacf-4eed9dc8bae0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9cda29438b944702b2dad445fa604697'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:18:04.191316 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-741b5c53-bab2-4654-bacf-4eed9dc8bae0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9cda29438b944702b2dad445fa604697'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:18:04.203251 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-741b5c53-bab2-4654-bacf-4eed9dc8bae0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9cda29438b944702b2dad445fa604697'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:18:04.215022 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1068/2108] 10.222.0.23 () {58 vars in 1223 bytes} [Sat Jun 18 05:18:03 2022] DELETE /identity/v3/users/9cda29438b944702b2dad445fa604697 => generated 0 bytes in 679 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:18:04.221127 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc81ccc-1847-4750-9a76-fe4d818bb7b5 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:18:04.225512 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc81ccc-1847-4750-9a76-fe4d818bb7b5 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:18:04.232140 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc81ccc-1847-4750-9a76-fe4d818bb7b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:18:04.284089 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-3dc81ccc-1847-4750-9a76-fe4d818bb7b5 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:18:04.285344 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-3dc81ccc-1847-4750-9a76-fe4d818bb7b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:18:04.432765 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1041/2109] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:18:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 18 05:18:04.445527 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741b5c53-bab2-4654-bacf-4eed9dc8bae0 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:18:04.449696 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-741b5c53-bab2-4654-bacf-4eed9dc8bae0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:18:04.581926 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741b5c53-bab2-4654-bacf-4eed9dc8bae0 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:18:04.597493 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ba0e1d-e3c6-459a-ab53-5adec42d68b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c4d7cde8f4ad9aa479bb09e325db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:18:04.598453 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ba0e1d-e3c6-459a-ab53-5adec42d68b2 service neutron] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:18:04.600232 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ba0e1d-e3c6-459a-ab53-5adec42d68b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:18:04.600733 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ba0e1d-e3c6-459a-ab53-5adec42d68b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:18:04.639234 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ba0e1d-e3c6-459a-ab53-5adec42d68b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:18:04.640998 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ba0e1d-e3c6-459a-ab53-5adec42d68b2 service neutron] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:18:04.664087 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1069/2110] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:18:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:18:04.921748 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7471139-f622-463c-a392-dc303538ca47 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:18:04.921748 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7471139-f622-463c-a392-dc303538ca47 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:18:04.921748 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7471139-f622-463c-a392-dc303538ca47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:18:05.007921 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-c7471139-f622-463c-a392-dc303538ca47 None None] MFA Rules not processed for user `2d63e07595cb4ebcafb24f6928f06f61`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:18:05.011651 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-c7471139-f622-463c-a392-dc303538ca47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:18:05.175686 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1042/2111] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:18:04 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:18:05.184937 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ba0e1d-e3c6-459a-ab53-5adec42d68b2 service neutron] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:18:05.186494 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-82ba0e1d-e3c6-459a-ab53-5adec42d68b2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:18:05.264536 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ba0e1d-e3c6-459a-ab53-5adec42d68b2 service neutron] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:18:05.268761 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82144a28-2773-4d51-bf08-e7262d2e59b1 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4638e5d1fa124667b07054a87c2d353e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:18:05.269640 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82144a28-2773-4d51-bf08-e7262d2e59b1 service glance] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:18:05.270005 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82144a28-2773-4d51-bf08-e7262d2e59b1 service glance] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:18:05.270340 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82144a28-2773-4d51-bf08-e7262d2e59b1 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:18:05.272658 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-82144a28-2773-4d51-bf08-e7262d2e59b1 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:18:05.350833 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82144a28-2773-4d51-bf08-e7262d2e59b1 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:18:05.352308 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82144a28-2773-4d51-bf08-e7262d2e59b1 service glance] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:18:05.363556 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1070/2112] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:18:05 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 18 05:18:05.370156 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7471139-f622-463c-a392-dc303538ca47 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:18:05.377606 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-c7471139-f622-463c-a392-dc303538ca47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:18:05.420361 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7471139-f622-463c-a392-dc303538ca47 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:18:05.428370 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252a6ccc-ac51-48fa-86a0-a00922b5a47a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:18:05.429521 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252a6ccc-ac51-48fa-86a0-a00922b5a47a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:18:05.429954 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252a6ccc-ac51-48fa-86a0-a00922b5a47a admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:18:05.430604 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252a6ccc-ac51-48fa-86a0-a00922b5a47a admin admin] PATH_INFO: `/v3/projects/bbe4dd7c2cd64627b0e0cee06a06275f` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:18:05.432566 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252a6ccc-ac51-48fa-86a0-a00922b5a47a admin admin] RBAC: Authorizing `identity:delete_project(project_id=bbe4dd7c2cd64627b0e0cee06a06275f)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:18:05.435070 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252a6ccc-ac51-48fa-86a0-a00922b5a47a admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:18:05.591670 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-252a6ccc-ac51-48fa-86a0-a00922b5a47a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bbe4dd7c2cd64627b0e0cee06a06275f', 'request_id': 'req-252a6ccc-ac51-48fa-86a0-a00922b5a47a'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:18:05.596203 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:18:05.597547 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-252a6ccc-ac51-48fa-86a0-a00922b5a47a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bbe4dd7c2cd64627b0e0cee06a06275f', 'request_id': 'req-252a6ccc-ac51-48fa-86a0-a00922b5a47a'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:18:05.683838 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cab717d-d6b7-48b4-a836-1d64dc8269b3 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:18:05.684665 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cab717d-d6b7-48b4-a836-1d64dc8269b3 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:18:05.684937 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cab717d-d6b7-48b4-a836-1d64dc8269b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:18:05.705095 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-8cab717d-d6b7-48b4-a836-1d64dc8269b3 None None] MFA Rules not processed for user `e817fafd0c1b4bf9944481cf116f1260`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:18:05.706895 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-8cab717d-d6b7-48b4-a836-1d64dc8269b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:18:05.718743 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:18:05.723457 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-252a6ccc-ac51-48fa-86a0-a00922b5a47a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bbe4dd7c2cd64627b0e0cee06a06275f', 'request_id': 'req-252a6ccc-ac51-48fa-86a0-a00922b5a47a'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:18:05.742759 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-252a6ccc-ac51-48fa-86a0-a00922b5a47a admin admin] The token cache is being invalidate because project bbe4dd7c2cd64627b0e0cee06a06275f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80468) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 18 05:18:05.743149 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-252a6ccc-ac51-48fa-86a0-a00922b5a47a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:18:05.744998 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-252a6ccc-ac51-48fa-86a0-a00922b5a47a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:18:05.747740 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1043/2113] 10.222.0.23 () {58 vars in 1232 bytes} [Sat Jun 18 05:18:05 2022] DELETE /identity/v3/projects/bbe4dd7c2cd64627b0e0cee06a06275f => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:18:05.752380 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252a6ccc-ac51-48fa-86a0-a00922b5a47a admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:18:05.754162 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-252a6ccc-ac51-48fa-86a0-a00922b5a47a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:18:05.787541 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1071/2114] 10.222.0.23 () {58 vars in 1009 bytes} [Sat Jun 18 05:18:05 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 18 05:18:05.830830 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252a6ccc-ac51-48fa-86a0-a00922b5a47a admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:18:05.833603 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cab717d-d6b7-48b4-a836-1d64dc8269b3 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:18:05.835092 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-8cab717d-d6b7-48b4-a836-1d64dc8269b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:18:05.836286 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e18c21-69cd-4833-b381-c19c802dd350 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:18:05.838235 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e18c21-69cd-4833-b381-c19c802dd350 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:18:05.838527 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e18c21-69cd-4833-b381-c19c802dd350 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:18:05.838804 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e18c21-69cd-4833-b381-c19c802dd350 admin admin] PATH_INFO: `/v3/users/08e5cdcfa3074129824b59621f440bb5` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:18:05.840044 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e18c21-69cd-4833-b381-c19c802dd350 admin admin] RBAC: Authorizing `identity:delete_user(user_id=08e5cdcfa3074129824b59621f440bb5)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:18:05.842762 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e18c21-69cd-4833-b381-c19c802dd350 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:18:05.906998 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cab717d-d6b7-48b4-a836-1d64dc8269b3 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:18:05.912151 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847f49a6-4188-428c-bfb6-7e200a6f512e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16b49e866398400580ae48aa5a96fc04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:18:05.912803 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847f49a6-4188-428c-bfb6-7e200a6f512e service swift] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:18:05.913125 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847f49a6-4188-428c-bfb6-7e200a6f512e service swift] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:18:05.913420 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847f49a6-4188-428c-bfb6-7e200a6f512e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:18:05.925905 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847f49a6-4188-428c-bfb6-7e200a6f512e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:18:05.927565 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847f49a6-4188-428c-bfb6-7e200a6f512e service swift] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:18:05.936304 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1072/2115] 10.222.0.23 () {60 vars in 1384 bytes} [Sat Jun 18 05:18:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 18 05:18:06.158382 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-57e18c21-69cd-4833-b381-c19c802dd350 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '08e5cdcfa3074129824b59621f440bb5'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:18:06.159344 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-57e18c21-69cd-4833-b381-c19c802dd350 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '08e5cdcfa3074129824b59621f440bb5'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:18:06.226843 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-57e18c21-69cd-4833-b381-c19c802dd350 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '08e5cdcfa3074129824b59621f440bb5'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:18:06.273684 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-57e18c21-69cd-4833-b381-c19c802dd350 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '08e5cdcfa3074129824b59621f440bb5'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:18:06.298690 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-57e18c21-69cd-4833-b381-c19c802dd350 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '08e5cdcfa3074129824b59621f440bb5'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:18:06.312703 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1044/2116] 10.222.0.23 () {58 vars in 1223 bytes} [Sat Jun 18 05:18:05 2022] DELETE /identity/v3/users/08e5cdcfa3074129824b59621f440bb5 => generated 0 bytes in 557 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:18:06.996250 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847f49a6-4188-428c-bfb6-7e200a6f512e service swift] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:18:07.010186 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847f49a6-4188-428c-bfb6-7e200a6f512e service swift] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:18:07.127995 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986098a8-0a69-463b-b98a-f476806db86a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:18:07.131494 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986098a8-0a69-463b-b98a-f476806db86a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:18:07.131494 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986098a8-0a69-463b-b98a-f476806db86a admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:18:07.131494 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986098a8-0a69-463b-b98a-f476806db86a admin admin] PATH_INFO: `/v3/projects/dabc40ef707d4a5fad758723cd168c5a` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:18:07.132945 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986098a8-0a69-463b-b98a-f476806db86a admin admin] RBAC: Authorizing `identity:delete_project(project_id=dabc40ef707d4a5fad758723cd168c5a)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:18:07.136080 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986098a8-0a69-463b-b98a-f476806db86a admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:18:07.350865 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-986098a8-0a69-463b-b98a-f476806db86a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dabc40ef707d4a5fad758723cd168c5a', 'request_id': 'req-986098a8-0a69-463b-b98a-f476806db86a'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:18:07.354626 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:18:07.367974 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-986098a8-0a69-463b-b98a-f476806db86a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dabc40ef707d4a5fad758723cd168c5a', 'request_id': 'req-986098a8-0a69-463b-b98a-f476806db86a'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:18:07.382430 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-986098a8-0a69-463b-b98a-f476806db86a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dabc40ef707d4a5fad758723cd168c5a', 'request_id': 'req-986098a8-0a69-463b-b98a-f476806db86a'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:18:07.476859 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-986098a8-0a69-463b-b98a-f476806db86a admin admin] The token cache is being invalidate because project dabc40ef707d4a5fad758723cd168c5a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80469) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 18 05:18:07.478381 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-986098a8-0a69-463b-b98a-f476806db86a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:18:07.483882 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-986098a8-0a69-463b-b98a-f476806db86a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:18:07.485553 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1073/2117] 10.222.0.23 () {58 vars in 1232 bytes} [Sat Jun 18 05:18:06 2022] DELETE /identity/v3/projects/dabc40ef707d4a5fad758723cd168c5a => generated 0 bytes in 490 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:18:07.490524 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00661928-0a9f-4647-9827-be76f79081d6 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:18:07.491071 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00661928-0a9f-4647-9827-be76f79081d6 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:18:07.491071 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00661928-0a9f-4647-9827-be76f79081d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:18:07.518547 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-00661928-0a9f-4647-9827-be76f79081d6 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:18:07.520987 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-00661928-0a9f-4647-9827-be76f79081d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:18:07.643805 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1045/2118] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:18:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:18:07.652290 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe62878-d8ae-4b51-92ca-6acaab5692d7 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:18:07.652580 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe62878-d8ae-4b51-92ca-6acaab5692d7 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:18:07.652856 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe62878-d8ae-4b51-92ca-6acaab5692d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:18:07.672861 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-cbe62878-d8ae-4b51-92ca-6acaab5692d7 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:18:07.674188 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-cbe62878-d8ae-4b51-92ca-6acaab5692d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:18:07.684247 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1074/2119] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:18:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:18:07.688518 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00661928-0a9f-4647-9827-be76f79081d6 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:18:07.698554 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:18:07.702235 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00661928-0a9f-4647-9827-be76f79081d6 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:18:07.707513 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c20657-5ed4-450c-b411-f65ac169e8dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:18:07.708176 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c20657-5ed4-450c-b411-f65ac169e8dd admin admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:18:07.708456 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c20657-5ed4-450c-b411-f65ac169e8dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:18:07.708729 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c20657-5ed4-450c-b411-f65ac169e8dd admin admin] PATH_INFO: `/v3/domains` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:18:07.709547 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c20657-5ed4-450c-b411-f65ac169e8dd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 05:18:07.709858 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c20657-5ed4-450c-b411-f65ac169e8dd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:18:07.711448 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c20657-5ed4-450c-b411-f65ac169e8dd admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:18:07.725575 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1046/2120] 10.222.0.23 () {58 vars in 1152 bytes} [Sat Jun 18 05:18:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 05:18:07.729990 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe62878-d8ae-4b51-92ca-6acaab5692d7 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:18:07.740096 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe62878-d8ae-4b51-92ca-6acaab5692d7 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:18:07.745004 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3abcbd1-689c-4e15-99a5-6e2a6a5e13e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:18:07.745704 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3abcbd1-689c-4e15-99a5-6e2a6a5e13e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:18:07.745986 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3abcbd1-689c-4e15-99a5-6e2a6a5e13e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:18:07.746275 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3abcbd1-689c-4e15-99a5-6e2a6a5e13e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:18:07.746956 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3abcbd1-689c-4e15-99a5-6e2a6a5e13e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:18:07.748865 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3abcbd1-689c-4e15-99a5-6e2a6a5e13e1 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:18:07.888359 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1075/2121] 10.222.0.23 () {60 vars in 1152 bytes} [Sat Jun 18 05:18:07 2022] POST /identity/v3/projects => generated 377 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:18:07.892563 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c20657-5ed4-450c-b411-f65ac169e8dd admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:18:07.910591 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c20657-5ed4-450c-b411-f65ac169e8dd admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:18:08.020034 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d242369-b95a-4a56-97f4-c56a5020bb9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:18:08.020716 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d242369-b95a-4a56-97f4-c56a5020bb9d admin admin] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:18:08.021003 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d242369-b95a-4a56-97f4-c56a5020bb9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:18:08.021318 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d242369-b95a-4a56-97f4-c56a5020bb9d admin admin] PATH_INFO: `/v3/users` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:18:08.022156 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d242369-b95a-4a56-97f4-c56a5020bb9d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:18:08.024250 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d242369-b95a-4a56-97f4-c56a5020bb9d admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:18:08.136025 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3abcbd1-689c-4e15-99a5-6e2a6a5e13e1 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:18:08.141975 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1047/2122] 10.222.0.23 () {60 vars in 1143 bytes} [Sat Jun 18 05:18:07 2022] POST /identity/v3/users => generated 289 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:18:08.151457 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3abcbd1-689c-4e15-99a5-6e2a6a5e13e1 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:18:08.157927 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ac48ba-89bb-49e0-b054-8f5458408289 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:18:08.158839 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ac48ba-89bb-49e0-b054-8f5458408289 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:18:08.159330 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ac48ba-89bb-49e0-b054-8f5458408289 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:18:08.159874 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ac48ba-89bb-49e0-b054-8f5458408289 admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:18:08.160905 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ac48ba-89bb-49e0-b054-8f5458408289 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:18:08.162939 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ac48ba-89bb-49e0-b054-8f5458408289 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:18:08.182651 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d242369-b95a-4a56-97f4-c56a5020bb9d admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:18:08.193972 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1076/2123] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:18:08 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:18:08.199212 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d242369-b95a-4a56-97f4-c56a5020bb9d admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:18:08.206869 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ee26a8-4ec6-445b-8f8c-9987e3e5d60e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:18:08.207822 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ee26a8-4ec6-445b-8f8c-9987e3e5d60e admin admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:18:08.208549 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ee26a8-4ec6-445b-8f8c-9987e3e5d60e admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:18:08.208989 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ee26a8-4ec6-445b-8f8c-9987e3e5d60e admin admin] PATH_INFO: `/v3/roles` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:18:08.210155 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ee26a8-4ec6-445b-8f8c-9987e3e5d60e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:18:08.214000 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ee26a8-4ec6-445b-8f8c-9987e3e5d60e admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:18:08.236700 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ac48ba-89bb-49e0-b054-8f5458408289 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:18:08.242508 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1048/2124] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:18:08 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:18:08.254289 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ac48ba-89bb-49e0-b054-8f5458408289 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:18:08.262276 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fc8ef6-45c8-4314-bce4-331c34203a31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:18:08.263498 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fc8ef6-45c8-4314-bce4-331c34203a31 admin admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:18:08.264024 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fc8ef6-45c8-4314-bce4-331c34203a31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:18:08.264487 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fc8ef6-45c8-4314-bce4-331c34203a31 admin admin] PATH_INFO: `/v3/projects/ca9e8c9edd354806ab27e97b73cc1565/users/676d7714daa2474ca11b2593231ad49d/roles/cbf5c4c26898413d93e73b46b41046a3` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:18:08.310970 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fc8ef6-45c8-4314-bce4-331c34203a31 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca9e8c9edd354806ab27e97b73cc1565, user_id=676d7714daa2474ca11b2593231ad49d, role_id=cbf5c4c26898413d93e73b46b41046a3)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:18:08.313181 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fc8ef6-45c8-4314-bce4-331c34203a31 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:18:08.427335 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1077/2125] 10.222.0.23 () {60 vars in 1482 bytes} [Sat Jun 18 05:18:08 2022] PUT /identity/v3/projects/ca9e8c9edd354806ab27e97b73cc1565/users/676d7714daa2474ca11b2593231ad49d/roles/cbf5c4c26898413d93e73b46b41046a3 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:18:08.439690 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f8e6d5-3856-4a60-8638-0ebe77925f73 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:18:08.440011 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f8e6d5-3856-4a60-8638-0ebe77925f73 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:18:08.440249 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f8e6d5-3856-4a60-8638-0ebe77925f73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:18:08.502465 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-09f8e6d5-3856-4a60-8638-0ebe77925f73 None None] MFA Rules not processed for user `676d7714daa2474ca11b2593231ad49d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:18:08.506055 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-09f8e6d5-3856-4a60-8638-0ebe77925f73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:18:08.642753 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fc8ef6-45c8-4314-bce4-331c34203a31 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:18:08.642977 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1049/2126] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:18:08 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:18:08.869508 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fc8ef6-45c8-4314-bce4-331c34203a31 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:18:08.886152 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21aa8dd6-c8de-4035-8ecc-1d7af90c6733 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:18:08.888591 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21aa8dd6-c8de-4035-8ecc-1d7af90c6733 admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:18:08.888591 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21aa8dd6-c8de-4035-8ecc-1d7af90c6733 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:18:08.888591 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21aa8dd6-c8de-4035-8ecc-1d7af90c6733 admin admin] PATH_INFO: `/v3/projects` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:18:08.891357 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21aa8dd6-c8de-4035-8ecc-1d7af90c6733 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:18:08.897321 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21aa8dd6-c8de-4035-8ecc-1d7af90c6733 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:18:09.027893 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f8e6d5-3856-4a60-8638-0ebe77925f73 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:18:09.039311 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1078/2127] 10.222.0.23 () {60 vars in 1152 bytes} [Sat Jun 18 05:18:08 2022] POST /identity/v3/projects => generated 377 bytes in 397 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 18 05:18:09.045055 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f8e6d5-3856-4a60-8638-0ebe77925f73 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:18:09.188377 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00967392-481f-4a50-86df-9c16682844de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:18:09.193542 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00967392-481f-4a50-86df-9c16682844de admin admin] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:18:09.194873 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00967392-481f-4a50-86df-9c16682844de admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:18:09.194873 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00967392-481f-4a50-86df-9c16682844de admin admin] PATH_INFO: `/v3/users` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:18:09.195325 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00967392-481f-4a50-86df-9c16682844de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:18:09.198994 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00967392-481f-4a50-86df-9c16682844de admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:18:09.211031 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:18:09.323418 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21aa8dd6-c8de-4035-8ecc-1d7af90c6733 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:18:09.329334 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1050/2128] 10.222.0.23 () {60 vars in 1143 bytes} [Sat Jun 18 05:18:09 2022] POST /identity/v3/users => generated 289 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:18:09.341019 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21aa8dd6-c8de-4035-8ecc-1d7af90c6733 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:18:09.355073 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e36628-ad42-40f0-ae02-c4252bb6fad8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:18:09.355869 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e36628-ad42-40f0-ae02-c4252bb6fad8 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:18:09.356107 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e36628-ad42-40f0-ae02-c4252bb6fad8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:18:09.356385 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e36628-ad42-40f0-ae02-c4252bb6fad8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:18:09.357180 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e36628-ad42-40f0-ae02-c4252bb6fad8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:18:09.361174 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e36628-ad42-40f0-ae02-c4252bb6fad8 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:18:09.404359 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00967392-481f-4a50-86df-9c16682844de admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:18:09.410875 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1079/2129] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:18:09 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 18 05:18:09.414560 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00967392-481f-4a50-86df-9c16682844de admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:18:09.429117 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8829be32-3697-4b76-8453-0bae5b2bb46c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:18:09.430093 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8829be32-3697-4b76-8453-0bae5b2bb46c admin admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:18:09.432613 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8829be32-3697-4b76-8453-0bae5b2bb46c admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:18:09.432613 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8829be32-3697-4b76-8453-0bae5b2bb46c admin admin] PATH_INFO: `/v3/roles` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:18:09.432613 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8829be32-3697-4b76-8453-0bae5b2bb46c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:18:09.439012 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8829be32-3697-4b76-8453-0bae5b2bb46c admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:18:09.464467 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1051/2130] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:18:09 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:18:09.465766 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e36628-ad42-40f0-ae02-c4252bb6fad8 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:18:09.484304 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e36628-ad42-40f0-ae02-c4252bb6fad8 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:18:09.495630 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56758077-fcf5-433b-a5a2-a54348771ec1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:18:09.498258 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56758077-fcf5-433b-a5a2-a54348771ec1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:18:09.498551 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56758077-fcf5-433b-a5a2-a54348771ec1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:18:09.498887 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56758077-fcf5-433b-a5a2-a54348771ec1 admin admin] PATH_INFO: `/v3/projects/a4525953a77a4f0aa54abf6510667ba9/users/aa9a76f0a6be442c9773ec6646ae52e7/roles/cbf5c4c26898413d93e73b46b41046a3` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:18:09.571559 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56758077-fcf5-433b-a5a2-a54348771ec1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4525953a77a4f0aa54abf6510667ba9, user_id=aa9a76f0a6be442c9773ec6646ae52e7, role_id=cbf5c4c26898413d93e73b46b41046a3)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:18:09.575041 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56758077-fcf5-433b-a5a2-a54348771ec1 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:18:09.685634 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1080/2131] 10.222.0.23 () {60 vars in 1482 bytes} [Sat Jun 18 05:18:09 2022] PUT /identity/v3/projects/a4525953a77a4f0aa54abf6510667ba9/users/aa9a76f0a6be442c9773ec6646ae52e7/roles/cbf5c4c26898413d93e73b46b41046a3 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:18:09.702108 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758efca1-9aa9-4936-97d5-ef3f8494894c None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:18:09.704513 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758efca1-9aa9-4936-97d5-ef3f8494894c None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:18:09.708253 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758efca1-9aa9-4936-97d5-ef3f8494894c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:18:09.771525 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-758efca1-9aa9-4936-97d5-ef3f8494894c None None] MFA Rules not processed for user `aa9a76f0a6be442c9773ec6646ae52e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:18:09.771525 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-758efca1-9aa9-4936-97d5-ef3f8494894c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:18:09.989016 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1052/2132] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:18:09 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:18:09.991769 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56758077-fcf5-433b-a5a2-a54348771ec1 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:18:10.086392 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=93,keystone:DELETE=2,keystone:INSERT=5 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:18:10.172047 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56758077-fcf5-433b-a5a2-a54348771ec1 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:18:10.177794 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a49313a-c39e-49df-bc50-31be275f2482 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:18:10.178608 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a49313a-c39e-49df-bc50-31be275f2482 admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:18:10.178896 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a49313a-c39e-49df-bc50-31be275f2482 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:18:10.179172 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a49313a-c39e-49df-bc50-31be275f2482 admin admin] PATH_INFO: `/v3/projects` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:18:10.181707 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a49313a-c39e-49df-bc50-31be275f2482 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:18:10.187385 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a49313a-c39e-49df-bc50-31be275f2482 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:18:10.478447 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1081/2133] 10.222.0.23 () {60 vars in 1152 bytes} [Sat Jun 18 05:18:09 2022] POST /identity/v3/projects => generated 377 bytes in 487 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:18:10.483164 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758efca1-9aa9-4936-97d5-ef3f8494894c None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:18:10.511515 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758efca1-9aa9-4936-97d5-ef3f8494894c None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:18:10.654201 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2a364b-6aa2-4a28-aad8-0d85b7bb765f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:18:10.654898 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2a364b-6aa2-4a28-aad8-0d85b7bb765f admin admin] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:18:10.655179 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2a364b-6aa2-4a28-aad8-0d85b7bb765f admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:18:10.655460 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2a364b-6aa2-4a28-aad8-0d85b7bb765f admin admin] PATH_INFO: `/v3/users` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:18:10.657258 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2a364b-6aa2-4a28-aad8-0d85b7bb765f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:18:10.658133 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2a364b-6aa2-4a28-aad8-0d85b7bb765f admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:18:10.806259 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1053/2134] 10.222.0.23 () {60 vars in 1143 bytes} [Sat Jun 18 05:18:10 2022] POST /identity/v3/users => generated 295 bytes in 324 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:18:10.810332 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a49313a-c39e-49df-bc50-31be275f2482 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:18:10.828967 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a49313a-c39e-49df-bc50-31be275f2482 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:18:10.850266 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52bdad8-587c-4a2e-8a35-9c0e166fd8ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:18:10.850868 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52bdad8-587c-4a2e-8a35-9c0e166fd8ca admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:18:10.851147 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52bdad8-587c-4a2e-8a35-9c0e166fd8ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:18:10.851418 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52bdad8-587c-4a2e-8a35-9c0e166fd8ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:18:10.852171 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52bdad8-587c-4a2e-8a35-9c0e166fd8ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:18:10.854045 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52bdad8-587c-4a2e-8a35-9c0e166fd8ca admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:18:10.893465 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1082/2135] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:18:10 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 18 05:18:10.897977 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2a364b-6aa2-4a28-aad8-0d85b7bb765f admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:18:10.929726 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2a364b-6aa2-4a28-aad8-0d85b7bb765f admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:18:10.949186 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6888ab11-6dc8-4b1f-9850-5882868ff7a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:18:10.950131 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6888ab11-6dc8-4b1f-9850-5882868ff7a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:18:10.950414 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6888ab11-6dc8-4b1f-9850-5882868ff7a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:18:10.950691 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6888ab11-6dc8-4b1f-9850-5882868ff7a9 admin admin] PATH_INFO: `/v3/projects/edce75da580b457a9543339f9b001e47/users/af1047a23f3d4fea80eb9aa42fc153c4/roles/21c65f2c51bd4dd283d13507d2714467` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:18:11.010732 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6888ab11-6dc8-4b1f-9850-5882868ff7a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=edce75da580b457a9543339f9b001e47, user_id=af1047a23f3d4fea80eb9aa42fc153c4, role_id=21c65f2c51bd4dd283d13507d2714467)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:18:11.018861 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6888ab11-6dc8-4b1f-9850-5882868ff7a9 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:18:11.097725 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1054/2136] 10.222.0.23 () {60 vars in 1482 bytes} [Sat Jun 18 05:18:10 2022] PUT /identity/v3/projects/edce75da580b457a9543339f9b001e47/users/af1047a23f3d4fea80eb9aa42fc153c4/roles/21c65f2c51bd4dd283d13507d2714467 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:18:11.099109 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52bdad8-587c-4a2e-8a35-9c0e166fd8ca admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:18:11.125872 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52bdad8-587c-4a2e-8a35-9c0e166fd8ca admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:18:11.266177 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7db7a2-c338-4db4-9acd-36a569009d83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:18:11.268002 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7db7a2-c338-4db4-9acd-36a569009d83 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:18:11.269710 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7db7a2-c338-4db4-9acd-36a569009d83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:18:11.270273 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7db7a2-c338-4db4-9acd-36a569009d83 admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:18:11.271463 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7db7a2-c338-4db4-9acd-36a569009d83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:18:11.287765 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7db7a2-c338-4db4-9acd-36a569009d83 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:18:11.318927 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1083/2137] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:18:11 2022] GET /identity/v3/roles => generated 1420 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:18:11.326462 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6888ab11-6dc8-4b1f-9850-5882868ff7a9 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:18:11.357597 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6888ab11-6dc8-4b1f-9850-5882868ff7a9 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:18:11.385241 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d584ca0-c84c-4b59-96b2-460f44110a11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:18:11.390418 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d584ca0-c84c-4b59-96b2-460f44110a11 admin admin] REQUEST_METHOD: `PUT` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:18:11.390857 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d584ca0-c84c-4b59-96b2-460f44110a11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:18:11.391153 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d584ca0-c84c-4b59-96b2-460f44110a11 admin admin] PATH_INFO: `/v3/projects/edce75da580b457a9543339f9b001e47/users/af1047a23f3d4fea80eb9aa42fc153c4/roles/21c65f2c51bd4dd283d13507d2714467` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:18:11.402475 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d584ca0-c84c-4b59-96b2-460f44110a11 admin admin] RBAC: Authorizing `identity:create_grant(project_id=edce75da580b457a9543339f9b001e47, user_id=af1047a23f3d4fea80eb9aa42fc153c4, role_id=21c65f2c51bd4dd283d13507d2714467)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:18:11.417932 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d584ca0-c84c-4b59-96b2-460f44110a11 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:18:11.467579 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1055/2138] 10.222.0.23 () {60 vars in 1482 bytes} [Sat Jun 18 05:18:11 2022] PUT /identity/v3/projects/edce75da580b457a9543339f9b001e47/users/af1047a23f3d4fea80eb9aa42fc153c4/roles/21c65f2c51bd4dd283d13507d2714467 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:18:11.481352 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f9ff14-e6da-4d14-8475-2c6c1a087ede None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:18:11.481859 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f9ff14-e6da-4d14-8475-2c6c1a087ede None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:18:11.481950 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f9ff14-e6da-4d14-8475-2c6c1a087ede None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:18:11.563177 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-c6f9ff14-e6da-4d14-8475-2c6c1a087ede None None] MFA Rules not processed for user `af1047a23f3d4fea80eb9aa42fc153c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:18:11.564416 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-c6f9ff14-e6da-4d14-8475-2c6c1a087ede None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:18:11.730235 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1084/2139] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:18:11 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:18:11.742527 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d584ca0-c84c-4b59-96b2-460f44110a11 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:18:11.746795 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-6d584ca0-c84c-4b59-96b2-460f44110a11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:18:11.786865 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:18:11.853528 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d584ca0-c84c-4b59-96b2-460f44110a11 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:18:11.858825 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196b6da7-3e9b-4bb3-ad35-b8cd0e9d0f54 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9de0428abc8400aa338aba4f7c34c94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:18:11.859735 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196b6da7-3e9b-4bb3-ad35-b8cd0e9d0f54 service cinder] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:18:11.860022 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196b6da7-3e9b-4bb3-ad35-b8cd0e9d0f54 service cinder] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:18:11.860300 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196b6da7-3e9b-4bb3-ad35-b8cd0e9d0f54 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:18:12.070555 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196b6da7-3e9b-4bb3-ad35-b8cd0e9d0f54 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:18:12.081244 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196b6da7-3e9b-4bb3-ad35-b8cd0e9d0f54 service cinder] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:18:12.133163 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1056/2140] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:18:11 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:18:15.929872 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f9ff14-e6da-4d14-8475-2c6c1a087ede None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:18:15.932374 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-c6f9ff14-e6da-4d14-8475-2c6c1a087ede None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:18:16.057636 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:18:16.115240 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f9ff14-e6da-4d14-8475-2c6c1a087ede None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:18:16.121493 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06301f5-b099-48c8-98b1-e5c61035d59d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:18:16.122751 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06301f5-b099-48c8-98b1-e5c61035d59d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:18:16.123195 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06301f5-b099-48c8-98b1-e5c61035d59d admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:18:16.123633 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06301f5-b099-48c8-98b1-e5c61035d59d admin admin] PATH_INFO: `/v3/users/04a9356f01c841ecb369984ecce9bc0e` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:18:16.125081 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06301f5-b099-48c8-98b1-e5c61035d59d admin admin] RBAC: Authorizing `identity:delete_user(user_id=04a9356f01c841ecb369984ecce9bc0e)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:18:16.127971 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06301f5-b099-48c8-98b1-e5c61035d59d admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:18:16.583598 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196b6da7-3e9b-4bb3-ad35-b8cd0e9d0f54 service cinder] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:18:16.598348 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196b6da7-3e9b-4bb3-ad35-b8cd0e9d0f54 service cinder] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:18:16.602429 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18a88a0-467b-4823-b359-b323169a071b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9de0428abc8400aa338aba4f7c34c94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:18:16.603135 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18a88a0-467b-4823-b359-b323169a071b service cinder] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:18:16.603426 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18a88a0-467b-4823-b359-b323169a071b service cinder] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:18:16.603709 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18a88a0-467b-4823-b359-b323169a071b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:18:16.627460 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-c06301f5-b099-48c8-98b1-e5c61035d59d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04a9356f01c841ecb369984ecce9bc0e'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:18:16.643780 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-c06301f5-b099-48c8-98b1-e5c61035d59d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04a9356f01c841ecb369984ecce9bc0e'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:18:16.644807 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-c06301f5-b099-48c8-98b1-e5c61035d59d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04a9356f01c841ecb369984ecce9bc0e'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:18:16.691591 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18a88a0-467b-4823-b359-b323169a071b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:18:16.693293 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18a88a0-467b-4823-b359-b323169a071b service cinder] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:18:16.706749 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1057/2141] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:18:16 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 18 05:18:16.716344 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-c06301f5-b099-48c8-98b1-e5c61035d59d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04a9356f01c841ecb369984ecce9bc0e'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:18:16.727740 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-c06301f5-b099-48c8-98b1-e5c61035d59d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04a9356f01c841ecb369984ecce9bc0e'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:18:16.735465 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1085/2142] 10.222.0.23 () {58 vars in 1223 bytes} [Sat Jun 18 05:18:15 2022] DELETE /identity/v3/users/04a9356f01c841ecb369984ecce9bc0e => generated 0 bytes in 806 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:18:16.740002 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e4aa33-0bb4-497e-88be-48b99652f1d9 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:18:16.740496 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e4aa33-0bb4-497e-88be-48b99652f1d9 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:18:16.740952 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e4aa33-0bb4-497e-88be-48b99652f1d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:18:16.782611 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-93e4aa33-0bb4-497e-88be-48b99652f1d9 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:18:16.788533 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-93e4aa33-0bb4-497e-88be-48b99652f1d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:18:16.877043 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1058/2143] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:18:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:18:16.886915 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06301f5-b099-48c8-98b1-e5c61035d59d admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:18:16.888630 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-c06301f5-b099-48c8-98b1-e5c61035d59d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:18:17.002837 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06301f5-b099-48c8-98b1-e5c61035d59d admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:18:17.006658 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78d914b-943c-48d4-885c-f29d2ac2d964 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c4d7cde8f4ad9aa479bb09e325db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:18:17.007570 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78d914b-943c-48d4-885c-f29d2ac2d964 service neutron] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:18:17.008052 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78d914b-943c-48d4-885c-f29d2ac2d964 service neutron] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:18:17.008333 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78d914b-943c-48d4-885c-f29d2ac2d964 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:18:17.019132 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78d914b-943c-48d4-885c-f29d2ac2d964 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:18:17.020683 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78d914b-943c-48d4-885c-f29d2ac2d964 service neutron] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:18:17.032197 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1086/2144] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:18:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:18:17.586773 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e4aa33-0bb4-497e-88be-48b99652f1d9 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:18:17.589252 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-93e4aa33-0bb4-497e-88be-48b99652f1d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:18:17.626337 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e4aa33-0bb4-497e-88be-48b99652f1d9 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:18:17.636768 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd76d31d-595a-42de-a6eb-a334c67d84fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:18:17.637764 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd76d31d-595a-42de-a6eb-a334c67d84fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:18:17.638160 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd76d31d-595a-42de-a6eb-a334c67d84fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:18:17.638518 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd76d31d-595a-42de-a6eb-a334c67d84fc admin admin] PATH_INFO: `/v3/projects/74f98b8de4f24490a553848191f1b3c7` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:18:17.639892 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd76d31d-595a-42de-a6eb-a334c67d84fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=74f98b8de4f24490a553848191f1b3c7)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:18:17.642930 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd76d31d-595a-42de-a6eb-a334c67d84fc admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:18:17.654503 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:18:17.761181 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-bd76d31d-595a-42de-a6eb-a334c67d84fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '74f98b8de4f24490a553848191f1b3c7', 'request_id': 'req-bd76d31d-595a-42de-a6eb-a334c67d84fc'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:18:17.770196 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-bd76d31d-595a-42de-a6eb-a334c67d84fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '74f98b8de4f24490a553848191f1b3c7', 'request_id': 'req-bd76d31d-595a-42de-a6eb-a334c67d84fc'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:18:17.848288 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-bd76d31d-595a-42de-a6eb-a334c67d84fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '74f98b8de4f24490a553848191f1b3c7', 'request_id': 'req-bd76d31d-595a-42de-a6eb-a334c67d84fc'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:18:17.870711 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-bd76d31d-595a-42de-a6eb-a334c67d84fc admin admin] The token cache is being invalidate because project 74f98b8de4f24490a553848191f1b3c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80468) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 18 05:18:17.871951 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-bd76d31d-595a-42de-a6eb-a334c67d84fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:18:17.873907 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-bd76d31d-595a-42de-a6eb-a334c67d84fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:18:17.879899 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1059/2145] 10.222.0.23 () {58 vars in 1232 bytes} [Sat Jun 18 05:18:17 2022] DELETE /identity/v3/projects/74f98b8de4f24490a553848191f1b3c7 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:18:17.886278 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8dbae2-ab9f-448a-b6b6-4b435f0388d8 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:18:17.886676 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8dbae2-ab9f-448a-b6b6-4b435f0388d8 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:18:17.886994 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8dbae2-ab9f-448a-b6b6-4b435f0388d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:18:17.908358 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-3d8dbae2-ab9f-448a-b6b6-4b435f0388d8 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:18:17.909688 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-3d8dbae2-ab9f-448a-b6b6-4b435f0388d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:18:17.986247 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1087/2146] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:18:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:18:17.992851 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00cb1b7-d7d7-46dd-bd90-e708fb86256c None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:18:17.993624 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00cb1b7-d7d7-46dd-bd90-e708fb86256c None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:18:17.994306 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00cb1b7-d7d7-46dd-bd90-e708fb86256c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:18:18.032902 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-c00cb1b7-d7d7-46dd-bd90-e708fb86256c None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:18:18.034075 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-c00cb1b7-d7d7-46dd-bd90-e708fb86256c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:18:18.042904 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1060/2147] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:18:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:18:18.047095 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8dbae2-ab9f-448a-b6b6-4b435f0388d8 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:18:18.061345 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8dbae2-ab9f-448a-b6b6-4b435f0388d8 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:18:18.067551 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3206baa-8aa3-415f-aaa6-34d584ced800 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:18:18.068410 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3206baa-8aa3-415f-aaa6-34d584ced800 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:18:18.068848 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3206baa-8aa3-415f-aaa6-34d584ced800 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:18:18.069275 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3206baa-8aa3-415f-aaa6-34d584ced800 admin admin] PATH_INFO: `/v3/domains` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:18:18.070190 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3206baa-8aa3-415f-aaa6-34d584ced800 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 05:18:18.070658 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3206baa-8aa3-415f-aaa6-34d584ced800 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:18:18.073541 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3206baa-8aa3-415f-aaa6-34d584ced800 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:18:18.090949 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1088/2148] 10.222.0.23 () {58 vars in 1152 bytes} [Sat Jun 18 05:18:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 05:18:18.092254 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00cb1b7-d7d7-46dd-bd90-e708fb86256c None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:18:18.101798 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00cb1b7-d7d7-46dd-bd90-e708fb86256c None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:18:18.108147 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ac9541-8a8f-419f-91a6-7eb5e0c0a706 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:18:18.108942 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ac9541-8a8f-419f-91a6-7eb5e0c0a706 admin admin] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:18:18.109230 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ac9541-8a8f-419f-91a6-7eb5e0c0a706 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:18:18.109542 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ac9541-8a8f-419f-91a6-7eb5e0c0a706 admin admin] PATH_INFO: `/v3/projects` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:18:18.118143 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ac9541-8a8f-419f-91a6-7eb5e0c0a706 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:18:18.118143 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ac9541-8a8f-419f-91a6-7eb5e0c0a706 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:18:18.139571 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:18:18.199461 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1061/2149] 10.222.0.23 () {60 vars in 1152 bytes} [Sat Jun 18 05:18:18 2022] POST /identity/v3/projects => generated 371 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 18 05:18:18.203257 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3206baa-8aa3-415f-aaa6-34d584ced800 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:18:18.215526 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3206baa-8aa3-415f-aaa6-34d584ced800 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:18:18.284620 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edda54d-0bab-4206-9d0e-35381cab8efc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:18:18.285637 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edda54d-0bab-4206-9d0e-35381cab8efc admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:18:18.286097 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edda54d-0bab-4206-9d0e-35381cab8efc admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:18:18.286737 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edda54d-0bab-4206-9d0e-35381cab8efc admin admin] PATH_INFO: `/v3/users` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:18:18.287598 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edda54d-0bab-4206-9d0e-35381cab8efc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:18:18.289908 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edda54d-0bab-4206-9d0e-35381cab8efc admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:18:18.407276 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1089/2150] 10.222.0.23 () {60 vars in 1143 bytes} [Sat Jun 18 05:18:18 2022] POST /identity/v3/users => generated 286 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:18:18.410996 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ac9541-8a8f-419f-91a6-7eb5e0c0a706 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:18:18.429050 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ac9541-8a8f-419f-91a6-7eb5e0c0a706 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:18:18.448850 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679dd131-c174-4d6e-9dbc-52a6ff5263fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:18:18.452443 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679dd131-c174-4d6e-9dbc-52a6ff5263fd admin admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:18:18.452443 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679dd131-c174-4d6e-9dbc-52a6ff5263fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:18:18.452443 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679dd131-c174-4d6e-9dbc-52a6ff5263fd admin admin] PATH_INFO: `/v3/roles` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:18:18.452443 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679dd131-c174-4d6e-9dbc-52a6ff5263fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:18:18.454849 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679dd131-c174-4d6e-9dbc-52a6ff5263fd admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:18:18.467225 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1062/2151] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:18:18 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:18:18.479082 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edda54d-0bab-4206-9d0e-35381cab8efc admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:18:18.490254 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edda54d-0bab-4206-9d0e-35381cab8efc admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:18:18.496566 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9459d857-28c3-4551-b19d-02671556f9f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:18:18.498155 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9459d857-28c3-4551-b19d-02671556f9f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:18:18.500490 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9459d857-28c3-4551-b19d-02671556f9f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:18:18.500877 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9459d857-28c3-4551-b19d-02671556f9f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:18:18.501692 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9459d857-28c3-4551-b19d-02671556f9f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:18:18.503354 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9459d857-28c3-4551-b19d-02671556f9f9 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:18:18.518014 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1090/2152] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:18:18 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:18:18.522123 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679dd131-c174-4d6e-9dbc-52a6ff5263fd admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:18:18.535945 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679dd131-c174-4d6e-9dbc-52a6ff5263fd admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:18:18.546978 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d4f4ce-3f19-4644-b024-bd8ce938b830 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:18:18.548402 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d4f4ce-3f19-4644-b024-bd8ce938b830 admin admin] REQUEST_METHOD: `PUT` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:18:18.548402 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d4f4ce-3f19-4644-b024-bd8ce938b830 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:18:18.548402 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d4f4ce-3f19-4644-b024-bd8ce938b830 admin admin] PATH_INFO: `/v3/projects/65a7c712ebe34d53b4acd7e8473c6d95/users/55bc735165cd4029bd5d581fd0187f9b/roles/cbf5c4c26898413d93e73b46b41046a3` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:18:18.569553 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d4f4ce-3f19-4644-b024-bd8ce938b830 admin admin] RBAC: Authorizing `identity:create_grant(project_id=65a7c712ebe34d53b4acd7e8473c6d95, user_id=55bc735165cd4029bd5d581fd0187f9b, role_id=cbf5c4c26898413d93e73b46b41046a3)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:18:18.573576 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d4f4ce-3f19-4644-b024-bd8ce938b830 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:18:18.638705 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1063/2153] 10.222.0.23 () {60 vars in 1482 bytes} [Sat Jun 18 05:18:18 2022] PUT /identity/v3/projects/65a7c712ebe34d53b4acd7e8473c6d95/users/55bc735165cd4029bd5d581fd0187f9b/roles/cbf5c4c26898413d93e73b46b41046a3 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:18:18.649039 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8918c0-6729-44ca-a649-72a17d79ac88 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:18:18.649325 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8918c0-6729-44ca-a649-72a17d79ac88 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:18:18.649711 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8918c0-6729-44ca-a649-72a17d79ac88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:18:18.669780 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-3a8918c0-6729-44ca-a649-72a17d79ac88 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:18:18.671177 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-3a8918c0-6729-44ca-a649-72a17d79ac88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:18:18.745662 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1091/2154] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:18:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:18:18.755777 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d4f4ce-3f19-4644-b024-bd8ce938b830 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:18:18.757546 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-77d4f4ce-3f19-4644-b024-bd8ce938b830 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:18:18.831760 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d4f4ce-3f19-4644-b024-bd8ce938b830 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:18:18.837744 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7707656-ce0c-428b-a6d2-4144a5f6f2f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c4d7cde8f4ad9aa479bb09e325db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:18:18.842492 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7707656-ce0c-428b-a6d2-4144a5f6f2f3 service neutron] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:18:18.842492 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7707656-ce0c-428b-a6d2-4144a5f6f2f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:18:18.842492 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7707656-ce0c-428b-a6d2-4144a5f6f2f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:18:18.850403 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7707656-ce0c-428b-a6d2-4144a5f6f2f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:18:18.851967 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7707656-ce0c-428b-a6d2-4144a5f6f2f3 service neutron] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:18:18.869165 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1064/2155] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:18:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:18:21.249580 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de6f621-168a-4e09-9174-bd4cf99cf3e6 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:18:21.250069 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de6f621-168a-4e09-9174-bd4cf99cf3e6 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:18:21.250257 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de6f621-168a-4e09-9174-bd4cf99cf3e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:18:21.269609 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-1de6f621-168a-4e09-9174-bd4cf99cf3e6 None None] MFA Rules not processed for user `113fba3872764b02bcbcc69299d8768f`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:18:21.270808 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-1de6f621-168a-4e09-9174-bd4cf99cf3e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:18:21.324965 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:18:21.335511 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1092/2156] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:18:21 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:18:21.595006 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7707656-ce0c-428b-a6d2-4144a5f6f2f3 service neutron] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:18:21.598520 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-f7707656-ce0c-428b-a6d2-4144a5f6f2f3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:18:21.709491 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:18:21.748645 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7707656-ce0c-428b-a6d2-4144a5f6f2f3 service neutron] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:18:21.757296 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b451270-5da6-4a1a-9981-20c46177a8e0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9de0428abc8400aa338aba4f7c34c94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:18:21.758135 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b451270-5da6-4a1a-9981-20c46177a8e0 service cinder] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:18:21.762119 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b451270-5da6-4a1a-9981-20c46177a8e0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:18:21.762119 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b451270-5da6-4a1a-9981-20c46177a8e0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:18:21.762119 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-3b451270-5da6-4a1a-9981-20c46177a8e0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:18:21.909976 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b451270-5da6-4a1a-9981-20c46177a8e0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:18:21.911952 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b451270-5da6-4a1a-9981-20c46177a8e0 service cinder] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:18:21.940110 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1065/2157] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:18:21 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:18:24.410324 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de6f621-168a-4e09-9174-bd4cf99cf3e6 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:18:24.411964 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-1de6f621-168a-4e09-9174-bd4cf99cf3e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:18:24.444849 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de6f621-168a-4e09-9174-bd4cf99cf3e6 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:18:24.456456 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d7610e-5988-4701-8a99-7d5ebb7f106c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:18:24.457789 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d7610e-5988-4701-8a99-7d5ebb7f106c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:18:24.458280 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d7610e-5988-4701-8a99-7d5ebb7f106c admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:18:24.458754 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d7610e-5988-4701-8a99-7d5ebb7f106c admin admin] PATH_INFO: `/v3/users/69bb7d5c41bb4e46b1b0bea2f7bad45f` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:18:24.461135 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d7610e-5988-4701-8a99-7d5ebb7f106c admin admin] RBAC: Authorizing `identity:delete_user(user_id=69bb7d5c41bb4e46b1b0bea2f7bad45f)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:18:24.465158 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d7610e-5988-4701-8a99-7d5ebb7f106c admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:18:24.702415 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-e8d7610e-5988-4701-8a99-7d5ebb7f106c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '69bb7d5c41bb4e46b1b0bea2f7bad45f'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:18:24.723811 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-e8d7610e-5988-4701-8a99-7d5ebb7f106c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '69bb7d5c41bb4e46b1b0bea2f7bad45f'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:18:24.725542 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-e8d7610e-5988-4701-8a99-7d5ebb7f106c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '69bb7d5c41bb4e46b1b0bea2f7bad45f'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:18:24.820080 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-e8d7610e-5988-4701-8a99-7d5ebb7f106c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '69bb7d5c41bb4e46b1b0bea2f7bad45f'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:18:24.831957 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-e8d7610e-5988-4701-8a99-7d5ebb7f106c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '69bb7d5c41bb4e46b1b0bea2f7bad45f'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:18:24.843464 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1093/2158] 10.222.0.23 () {58 vars in 1223 bytes} [Sat Jun 18 05:18:24 2022] DELETE /identity/v3/users/69bb7d5c41bb4e46b1b0bea2f7bad45f => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:18:24.853279 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d764c053-5839-412f-b66a-4a1adbe1baaa None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:18:24.853746 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d764c053-5839-412f-b66a-4a1adbe1baaa None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:18:24.853933 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d764c053-5839-412f-b66a-4a1adbe1baaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:18:24.900680 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-d764c053-5839-412f-b66a-4a1adbe1baaa None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:18:24.906178 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-d764c053-5839-412f-b66a-4a1adbe1baaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:18:25.028066 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1066/2159] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:18:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:18:25.036132 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d7610e-5988-4701-8a99-7d5ebb7f106c admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:18:25.046241 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-e8d7610e-5988-4701-8a99-7d5ebb7f106c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:18:25.186782 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d7610e-5988-4701-8a99-7d5ebb7f106c admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:18:25.191101 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3734575-419d-4c6d-ba82-0454e754d238 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c4d7cde8f4ad9aa479bb09e325db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:18:25.191969 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3734575-419d-4c6d-ba82-0454e754d238 service neutron] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:18:25.193088 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3734575-419d-4c6d-ba82-0454e754d238 service neutron] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:18:25.193658 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3734575-419d-4c6d-ba82-0454e754d238 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:18:25.204341 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3734575-419d-4c6d-ba82-0454e754d238 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:18:25.206360 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3734575-419d-4c6d-ba82-0454e754d238 service neutron] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:18:25.226881 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1094/2160] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:18:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:18:25.910785 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d764c053-5839-412f-b66a-4a1adbe1baaa None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:18:25.913230 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-d764c053-5839-412f-b66a-4a1adbe1baaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:18:25.938352 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d764c053-5839-412f-b66a-4a1adbe1baaa None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:18:25.953098 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d24131-65bc-42aa-9bf9-c82b15acdf44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:18:25.954201 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d24131-65bc-42aa-9bf9-c82b15acdf44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:18:25.955864 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d24131-65bc-42aa-9bf9-c82b15acdf44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:18:25.955864 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d24131-65bc-42aa-9bf9-c82b15acdf44 admin admin] PATH_INFO: `/v3/projects/b9f83a8cb78d4c7face46c6d787bfc5d` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:18:25.960273 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d24131-65bc-42aa-9bf9-c82b15acdf44 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9f83a8cb78d4c7face46c6d787bfc5d)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:18:25.964633 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d24131-65bc-42aa-9bf9-c82b15acdf44 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:18:26.176805 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-49d24131-65bc-42aa-9bf9-c82b15acdf44 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9f83a8cb78d4c7face46c6d787bfc5d', 'request_id': 'req-49d24131-65bc-42aa-9bf9-c82b15acdf44'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:18:26.202798 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-49d24131-65bc-42aa-9bf9-c82b15acdf44 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9f83a8cb78d4c7face46c6d787bfc5d', 'request_id': 'req-49d24131-65bc-42aa-9bf9-c82b15acdf44'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:18:26.287860 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-49d24131-65bc-42aa-9bf9-c82b15acdf44 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9f83a8cb78d4c7face46c6d787bfc5d', 'request_id': 'req-49d24131-65bc-42aa-9bf9-c82b15acdf44'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:18:26.339208 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-49d24131-65bc-42aa-9bf9-c82b15acdf44 admin admin] The token cache is being invalidate because project b9f83a8cb78d4c7face46c6d787bfc5d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80468) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 18 05:18:26.340362 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-49d24131-65bc-42aa-9bf9-c82b15acdf44 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:18:26.340571 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-49d24131-65bc-42aa-9bf9-c82b15acdf44 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:18:26.342208 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1067/2161] 10.222.0.23 () {58 vars in 1232 bytes} [Sat Jun 18 05:18:25 2022] DELETE /identity/v3/projects/b9f83a8cb78d4c7face46c6d787bfc5d => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:18:26.347148 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00111575-1720-489b-a073-5b55dacf43f3 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:18:26.347653 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00111575-1720-489b-a073-5b55dacf43f3 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:18:26.347770 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00111575-1720-489b-a073-5b55dacf43f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:18:26.370221 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-00111575-1720-489b-a073-5b55dacf43f3 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:18:26.371486 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-00111575-1720-489b-a073-5b55dacf43f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:18:26.423644 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80551d0-a5c5-46c1-95bf-8213166d1890 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:18:26.423941 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80551d0-a5c5-46c1-95bf-8213166d1890 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:18:26.424256 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80551d0-a5c5-46c1-95bf-8213166d1890 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:18:26.449968 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:18:26.463741 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-e80551d0-a5c5-46c1-95bf-8213166d1890 None None] MFA Rules not processed for user `55bc735165cd4029bd5d581fd0187f9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:18:26.465545 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-e80551d0-a5c5-46c1-95bf-8213166d1890 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:18:26.516735 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1095/2162] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:18:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:18:26.521166 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557b824e-73e8-4bb3-b62d-59270e4dcc74 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:18:26.521671 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557b824e-73e8-4bb3-b62d-59270e4dcc74 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:18:26.522008 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557b824e-73e8-4bb3-b62d-59270e4dcc74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:18:26.545623 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-557b824e-73e8-4bb3-b62d-59270e4dcc74 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:18:26.547623 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-557b824e-73e8-4bb3-b62d-59270e4dcc74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:18:26.549019 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1068/2163] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:18:26 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:18:26.553416 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80551d0-a5c5-46c1-95bf-8213166d1890 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:18:26.554756 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-e80551d0-a5c5-46c1-95bf-8213166d1890 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:18:26.560038 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1096/2164] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:18:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 18 05:18:26.561093 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557b824e-73e8-4bb3-b62d-59270e4dcc74 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:18:26.567104 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:18:26.577434 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80551d0-a5c5-46c1-95bf-8213166d1890 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:18:26.577565 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557b824e-73e8-4bb3-b62d-59270e4dcc74 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:18:26.579137 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76f4fa0-6593-495e-aa29-aebf58f77bfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:18:26.579916 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76f4fa0-6593-495e-aa29-aebf58f77bfb admin admin] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:18:26.580203 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76f4fa0-6593-495e-aa29-aebf58f77bfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:18:26.580471 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76f4fa0-6593-495e-aa29-aebf58f77bfb admin admin] PATH_INFO: `/v3/projects` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:18:26.581090 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4c4c06-3066-4d05-9c58-791b4a6919af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:18:26.581213 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76f4fa0-6593-495e-aa29-aebf58f77bfb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:18:26.581867 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4c4c06-3066-4d05-9c58-791b4a6919af admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:18:26.582219 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4c4c06-3066-4d05-9c58-791b4a6919af admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:18:26.582624 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4c4c06-3066-4d05-9c58-791b4a6919af admin admin] PATH_INFO: `/v3/domains` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:18:26.583920 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76f4fa0-6593-495e-aa29-aebf58f77bfb admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:18:26.584037 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4c4c06-3066-4d05-9c58-791b4a6919af admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 05:18:26.584037 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4c4c06-3066-4d05-9c58-791b4a6919af admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:18:26.585766 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4c4c06-3066-4d05-9c58-791b4a6919af admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:18:26.604639 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1097/2165] 10.222.0.23 () {58 vars in 1152 bytes} [Sat Jun 18 05:18:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 05:18:26.605917 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4c4c06-3066-4d05-9c58-791b4a6919af admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:18:26.616821 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4c4c06-3066-4d05-9c58-791b4a6919af admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:18:26.622676 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ce651c-56ae-4f04-a364-7a45f7cb517c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:18:26.623714 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ce651c-56ae-4f04-a364-7a45f7cb517c admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:18:26.630583 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ce651c-56ae-4f04-a364-7a45f7cb517c admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:18:26.630583 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ce651c-56ae-4f04-a364-7a45f7cb517c admin admin] PATH_INFO: `/v3/projects` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:18:26.630583 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ce651c-56ae-4f04-a364-7a45f7cb517c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:18:26.630583 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ce651c-56ae-4f04-a364-7a45f7cb517c admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:18:26.686428 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1069/2166] 10.222.0.23 () {60 vars in 1152 bytes} [Sat Jun 18 05:18:26 2022] POST /identity/v3/projects => generated 371 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 18 05:18:26.690183 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76f4fa0-6593-495e-aa29-aebf58f77bfb admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:18:26.767818 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1098/2167] 10.222.0.23 () {60 vars in 1152 bytes} [Sat Jun 18 05:18:26 2022] POST /identity/v3/projects => generated 371 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:18:26.772660 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ce651c-56ae-4f04-a364-7a45f7cb517c admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:18:26.815880 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76f4fa0-6593-495e-aa29-aebf58f77bfb admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:18:26.902316 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ce651c-56ae-4f04-a364-7a45f7cb517c admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:18:26.924145 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caf25c0-106a-440c-bc36-9f3c8e2ba827 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:18:26.936433 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caf25c0-106a-440c-bc36-9f3c8e2ba827 admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:18:26.936433 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caf25c0-106a-440c-bc36-9f3c8e2ba827 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:18:26.936433 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caf25c0-106a-440c-bc36-9f3c8e2ba827 admin admin] PATH_INFO: `/v3/users` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:18:26.936433 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3caf25c0-106a-440c-bc36-9f3c8e2ba827 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:18:26.936433 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3caf25c0-106a-440c-bc36-9f3c8e2ba827 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:18:26.965690 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1874b3b3-9bc7-456b-9f70-52267eda41c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:18:26.966529 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1874b3b3-9bc7-456b-9f70-52267eda41c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:18:26.967005 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1874b3b3-9bc7-456b-9f70-52267eda41c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:18:26.967444 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1874b3b3-9bc7-456b-9f70-52267eda41c9 admin admin] PATH_INFO: `/v3/users` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:18:26.968305 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1874b3b3-9bc7-456b-9f70-52267eda41c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:18:26.971287 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1874b3b3-9bc7-456b-9f70-52267eda41c9 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:18:27.105632 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1070/2168] 10.222.0.23 () {60 vars in 1143 bytes} [Sat Jun 18 05:18:26 2022] POST /identity/v3/users => generated 292 bytes in 416 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:18:27.107159 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1874b3b3-9bc7-456b-9f70-52267eda41c9 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:18:27.121747 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1099/2169] 10.222.0.23 () {60 vars in 1143 bytes} [Sat Jun 18 05:18:26 2022] POST /identity/v3/users => generated 286 bytes in 349 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:18:27.123144 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caf25c0-106a-440c-bc36-9f3c8e2ba827 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:18:27.124401 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1874b3b3-9bc7-456b-9f70-52267eda41c9 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:18:27.147274 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d49bd94-50ae-4831-b63c-f9d11c8b966c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:18:27.147650 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caf25c0-106a-440c-bc36-9f3c8e2ba827 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:18:27.147985 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d49bd94-50ae-4831-b63c-f9d11c8b966c admin admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:18:27.148231 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d49bd94-50ae-4831-b63c-f9d11c8b966c admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:18:27.149898 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d49bd94-50ae-4831-b63c-f9d11c8b966c admin admin] PATH_INFO: `/v3/roles` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:18:27.151726 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d49bd94-50ae-4831-b63c-f9d11c8b966c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:18:27.154005 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d49bd94-50ae-4831-b63c-f9d11c8b966c admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:18:27.169210 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f048f8-f15c-4d56-aa2e-5c10f4914f94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:18:27.172229 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f048f8-f15c-4d56-aa2e-5c10f4914f94 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:18:27.172631 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f048f8-f15c-4d56-aa2e-5c10f4914f94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:18:27.175251 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f048f8-f15c-4d56-aa2e-5c10f4914f94 admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:18:27.175989 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f048f8-f15c-4d56-aa2e-5c10f4914f94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:18:27.179525 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f048f8-f15c-4d56-aa2e-5c10f4914f94 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:18:27.185028 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1071/2170] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:18:27 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:18:27.189685 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d49bd94-50ae-4831-b63c-f9d11c8b966c admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:18:27.207648 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1100/2171] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:18:27 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:18:27.211586 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f048f8-f15c-4d56-aa2e-5c10f4914f94 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:18:27.229925 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f048f8-f15c-4d56-aa2e-5c10f4914f94 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:18:27.232778 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d49bd94-50ae-4831-b63c-f9d11c8b966c admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:18:27.242574 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6b83df-c2ca-406e-9d05-9c66eecd3f0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:18:27.244364 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6b83df-c2ca-406e-9d05-9c66eecd3f0b admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:18:27.245001 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6b83df-c2ca-406e-9d05-9c66eecd3f0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:18:27.245580 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6b83df-c2ca-406e-9d05-9c66eecd3f0b admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:18:27.247490 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6b83df-c2ca-406e-9d05-9c66eecd3f0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:18:27.247620 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db4ca58-df19-4a57-85d0-4fd0cdd6427a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:18:27.248645 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db4ca58-df19-4a57-85d0-4fd0cdd6427a admin admin] REQUEST_METHOD: `PUT` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:18:27.249592 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db4ca58-df19-4a57-85d0-4fd0cdd6427a admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:18:27.251179 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db4ca58-df19-4a57-85d0-4fd0cdd6427a admin admin] PATH_INFO: `/v3/projects/a42225556e894a4b83bf66b4a3cb2522/users/2d5c6fe0cb7e4da69cb69d8de910a82d/roles/21c65f2c51bd4dd283d13507d2714467` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:18:27.252944 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6b83df-c2ca-406e-9d05-9c66eecd3f0b admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:18:27.276929 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1101/2172] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:18:27 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:18:27.287887 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db4ca58-df19-4a57-85d0-4fd0cdd6427a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a42225556e894a4b83bf66b4a3cb2522, user_id=2d5c6fe0cb7e4da69cb69d8de910a82d, role_id=21c65f2c51bd4dd283d13507d2714467)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:18:27.292874 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db4ca58-df19-4a57-85d0-4fd0cdd6427a admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:18:27.297649 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6b83df-c2ca-406e-9d05-9c66eecd3f0b admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:18:27.305796 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:18:27.331718 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6b83df-c2ca-406e-9d05-9c66eecd3f0b admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:18:27.345799 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a010d57f-79c0-41e1-af45-08f35f1d9395 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:18:27.346943 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a010d57f-79c0-41e1-af45-08f35f1d9395 admin admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:18:27.347398 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a010d57f-79c0-41e1-af45-08f35f1d9395 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:18:27.353758 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a010d57f-79c0-41e1-af45-08f35f1d9395 admin admin] PATH_INFO: `/v3/projects/a63728833c434410864d0d7431a9fac2/users/4147695d07c24202b096fb7da07ecf37/roles/cbf5c4c26898413d93e73b46b41046a3` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:18:27.404732 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a010d57f-79c0-41e1-af45-08f35f1d9395 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a63728833c434410864d0d7431a9fac2, user_id=4147695d07c24202b096fb7da07ecf37, role_id=cbf5c4c26898413d93e73b46b41046a3)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:18:27.408696 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a010d57f-79c0-41e1-af45-08f35f1d9395 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:18:27.417118 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1072/2173] 10.222.0.23 () {60 vars in 1482 bytes} [Sat Jun 18 05:18:27 2022] PUT /identity/v3/projects/a42225556e894a4b83bf66b4a3cb2522/users/2d5c6fe0cb7e4da69cb69d8de910a82d/roles/21c65f2c51bd4dd283d13507d2714467 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:18:27.417118 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db4ca58-df19-4a57-85d0-4fd0cdd6427a admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:18:27.417118 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-6db4ca58-df19-4a57-85d0-4fd0cdd6427a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:18:27.496356 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db4ca58-df19-4a57-85d0-4fd0cdd6427a admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:18:27.504208 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b4f714-90cd-4ffe-baed-4661417ffa0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:18:27.505320 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b4f714-90cd-4ffe-baed-4661417ffa0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:18:27.505741 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b4f714-90cd-4ffe-baed-4661417ffa0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:18:27.506087 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b4f714-90cd-4ffe-baed-4661417ffa0b admin admin] PATH_INFO: `/v3/users/d2368c56c4f448ea988d25f4f1325527` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:18:27.508260 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b4f714-90cd-4ffe-baed-4661417ffa0b admin admin] RBAC: Authorizing `identity:delete_user(user_id=d2368c56c4f448ea988d25f4f1325527)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:18:27.510320 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1102/2174] 10.222.0.23 () {60 vars in 1482 bytes} [Sat Jun 18 05:18:27 2022] PUT /identity/v3/projects/a63728833c434410864d0d7431a9fac2/users/4147695d07c24202b096fb7da07ecf37/roles/cbf5c4c26898413d93e73b46b41046a3 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:18:27.511530 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b4f714-90cd-4ffe-baed-4661417ffa0b admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:18:27.514026 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1bc6e2-8e4a-4bef-a6f3-348ace79ad28 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:18:27.514907 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1bc6e2-8e4a-4bef-a6f3-348ace79ad28 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:18:27.528977 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1bc6e2-8e4a-4bef-a6f3-348ace79ad28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:18:27.571206 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-5e1bc6e2-8e4a-4bef-a6f3-348ace79ad28 None None] MFA Rules not processed for user `e817fafd0c1b4bf9944481cf116f1260`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:18:27.573055 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-5e1bc6e2-8e4a-4bef-a6f3-348ace79ad28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:18:27.645032 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1103/2175] 10.222.0.23 () {58 vars in 1009 bytes} [Sat Jun 18 05:18:27 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:18:27.646040 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1bc6e2-8e4a-4bef-a6f3-348ace79ad28 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:18:27.647434 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-5e1bc6e2-8e4a-4bef-a6f3-348ace79ad28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:18:27.669098 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:18:27.721882 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1bc6e2-8e4a-4bef-a6f3-348ace79ad28 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:18:27.728431 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c3c52a-06ee-401f-be67-8be7e2b842de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:18:27.730076 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c3c52a-06ee-401f-be67-8be7e2b842de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:18:27.730355 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c3c52a-06ee-401f-be67-8be7e2b842de admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:18:27.730641 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c3c52a-06ee-401f-be67-8be7e2b842de admin admin] PATH_INFO: `/v3/users/676d7714daa2474ca11b2593231ad49d` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:18:27.733719 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c3c52a-06ee-401f-be67-8be7e2b842de admin admin] RBAC: Authorizing `identity:delete_user(user_id=676d7714daa2474ca11b2593231ad49d)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:18:27.735896 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c3c52a-06ee-401f-be67-8be7e2b842de admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:18:27.739195 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-65b4f714-90cd-4ffe-baed-4661417ffa0b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd2368c56c4f448ea988d25f4f1325527'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:18:27.740010 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-65b4f714-90cd-4ffe-baed-4661417ffa0b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd2368c56c4f448ea988d25f4f1325527'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:18:27.796733 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-65b4f714-90cd-4ffe-baed-4661417ffa0b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd2368c56c4f448ea988d25f4f1325527'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:18:27.818844 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-65b4f714-90cd-4ffe-baed-4661417ffa0b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd2368c56c4f448ea988d25f4f1325527'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:18:27.835991 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-65b4f714-90cd-4ffe-baed-4661417ffa0b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd2368c56c4f448ea988d25f4f1325527'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:18:27.842782 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1073/2176] 10.222.0.23 () {58 vars in 1223 bytes} [Sat Jun 18 05:18:27 2022] DELETE /identity/v3/users/d2368c56c4f448ea988d25f4f1325527 => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:18:27.843776 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b4f714-90cd-4ffe-baed-4661417ffa0b admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:18:27.845000 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-65b4f714-90cd-4ffe-baed-4661417ffa0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:18:27.926661 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-27c3c52a-06ee-401f-be67-8be7e2b842de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '676d7714daa2474ca11b2593231ad49d'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:18:27.930000 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:18:27.935579 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b4f714-90cd-4ffe-baed-4661417ffa0b admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:18:27.941303 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0122f09-68ef-4bb2-9330-d3183a5745c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:18:27.942208 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0122f09-68ef-4bb2-9330-d3183a5745c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:18:27.942512 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0122f09-68ef-4bb2-9330-d3183a5745c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:18:27.943321 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0122f09-68ef-4bb2-9330-d3183a5745c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:18:27.945685 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0122f09-68ef-4bb2-9330-d3183a5745c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:18:27.945812 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-27c3c52a-06ee-401f-be67-8be7e2b842de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '676d7714daa2474ca11b2593231ad49d'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:18:27.947508 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0122f09-68ef-4bb2-9330-d3183a5745c5 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:18:27.947630 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-27c3c52a-06ee-401f-be67-8be7e2b842de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '676d7714daa2474ca11b2593231ad49d'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:18:27.961031 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1074/2177] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:18:27 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:18:27.962696 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2a60b5-2a80-48f9-8ff0-bf12b966d0b5 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:18:27.963034 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2a60b5-2a80-48f9-8ff0-bf12b966d0b5 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:18:27.963352 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2a60b5-2a80-48f9-8ff0-bf12b966d0b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:18:27.989920 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-fa2a60b5-2a80-48f9-8ff0-bf12b966d0b5 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:18:27.991364 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-fa2a60b5-2a80-48f9-8ff0-bf12b966d0b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:18:27.996713 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-27c3c52a-06ee-401f-be67-8be7e2b842de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '676d7714daa2474ca11b2593231ad49d'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:18:28.010090 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1075/2178] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:18:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:18:28.011081 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2a60b5-2a80-48f9-8ff0-bf12b966d0b5 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:18:28.013535 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-27c3c52a-06ee-401f-be67-8be7e2b842de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '676d7714daa2474ca11b2593231ad49d'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:18:28.029903 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1104/2179] 10.222.0.23 () {58 vars in 1223 bytes} [Sat Jun 18 05:18:27 2022] DELETE /identity/v3/users/676d7714daa2474ca11b2593231ad49d => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:18:28.031500 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affc5a8e-fac5-4388-b3d8-3a9b0d80dab0 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:18:28.032126 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affc5a8e-fac5-4388-b3d8-3a9b0d80dab0 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:18:28.032656 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affc5a8e-fac5-4388-b3d8-3a9b0d80dab0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:18:28.043989 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-fa2a60b5-2a80-48f9-8ff0-bf12b966d0b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:18:28.063131 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-affc5a8e-fac5-4388-b3d8-3a9b0d80dab0 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:18:28.064203 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-affc5a8e-fac5-4388-b3d8-3a9b0d80dab0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:18:28.210233 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1105/2180] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:18:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:18:28.211275 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affc5a8e-fac5-4388-b3d8-3a9b0d80dab0 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:18:28.212702 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2a60b5-2a80-48f9-8ff0-bf12b966d0b5 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:18:28.215222 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-affc5a8e-fac5-4388-b3d8-3a9b0d80dab0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:18:28.236411 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affc5a8e-fac5-4388-b3d8-3a9b0d80dab0 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:18:28.242016 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea47e1c4-fb69-4ec8-a6d0-a5b6f78de9eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:18:28.243127 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea47e1c4-fb69-4ec8-a6d0-a5b6f78de9eb admin admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:18:28.247570 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea47e1c4-fb69-4ec8-a6d0-a5b6f78de9eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:18:28.248032 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea47e1c4-fb69-4ec8-a6d0-a5b6f78de9eb admin admin] PATH_INFO: `/v3/projects/a42225556e894a4b83bf66b4a3cb2522/users/2d5c6fe0cb7e4da69cb69d8de910a82d/roles/21c65f2c51bd4dd283d13507d2714467` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:18:28.256397 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea47e1c4-fb69-4ec8-a6d0-a5b6f78de9eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=a42225556e894a4b83bf66b4a3cb2522, user_id=2d5c6fe0cb7e4da69cb69d8de910a82d, role_id=21c65f2c51bd4dd283d13507d2714467)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:18:28.261245 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea47e1c4-fb69-4ec8-a6d0-a5b6f78de9eb admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:18:28.280402 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1106/2181] 10.222.0.23 () {60 vars in 1482 bytes} [Sat Jun 18 05:18:28 2022] PUT /identity/v3/projects/a42225556e894a4b83bf66b4a3cb2522/users/2d5c6fe0cb7e4da69cb69d8de910a82d/roles/21c65f2c51bd4dd283d13507d2714467 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:18:28.281708 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761f6111-e6ef-4ae2-98f0-106cc9bb5db5 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:18:28.281990 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761f6111-e6ef-4ae2-98f0-106cc9bb5db5 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:18:28.282279 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761f6111-e6ef-4ae2-98f0-106cc9bb5db5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:18:28.328814 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-761f6111-e6ef-4ae2-98f0-106cc9bb5db5 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:18:28.331508 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-761f6111-e6ef-4ae2-98f0-106cc9bb5db5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:18:28.339755 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73ea798-4cde-42c0-9543-63d854237cf0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16b49e866398400580ae48aa5a96fc04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:18:28.340421 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73ea798-4cde-42c0-9543-63d854237cf0 service swift] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:18:28.340709 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73ea798-4cde-42c0-9543-63d854237cf0 service swift] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:18:28.340987 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73ea798-4cde-42c0-9543-63d854237cf0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:18:28.347472 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-b73ea798-4cde-42c0-9543-63d854237cf0 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:18:28.373078 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:18:28.429366 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73ea798-4cde-42c0-9543-63d854237cf0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:18:28.433142 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73ea798-4cde-42c0-9543-63d854237cf0 service swift] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:18:28.441429 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1107/2182] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:18:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:18:28.442582 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761f6111-e6ef-4ae2-98f0-106cc9bb5db5 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:18:28.454267 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-761f6111-e6ef-4ae2-98f0-106cc9bb5db5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:18:28.462372 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1076/2183] 10.222.0.23 () {60 vars in 1384 bytes} [Sat Jun 18 05:18:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 18 05:18:28.464297 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73ea798-4cde-42c0-9543-63d854237cf0 service swift] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:18:28.470597 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-b73ea798-4cde-42c0-9543-63d854237cf0 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:18:28.530958 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:18:28.538161 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761f6111-e6ef-4ae2-98f0-106cc9bb5db5 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:18:28.544410 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230d518b-58e5-4717-b98f-ceaf82474c6d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c4d7cde8f4ad9aa479bb09e325db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:18:28.545255 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230d518b-58e5-4717-b98f-ceaf82474c6d service neutron] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:18:28.545631 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230d518b-58e5-4717-b98f-ceaf82474c6d service neutron] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:18:28.545955 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230d518b-58e5-4717-b98f-ceaf82474c6d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:18:28.557888 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230d518b-58e5-4717-b98f-ceaf82474c6d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:18:28.559360 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230d518b-58e5-4717-b98f-ceaf82474c6d service neutron] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:18:28.560680 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73ea798-4cde-42c0-9543-63d854237cf0 service swift] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:18:28.565185 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8066c9fb-5fc2-4886-9161-4db321ad14e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c4d7cde8f4ad9aa479bb09e325db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:18:28.565185 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8066c9fb-5fc2-4886-9161-4db321ad14e4 service neutron] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:18:28.567952 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8066c9fb-5fc2-4886-9161-4db321ad14e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:18:28.567952 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8066c9fb-5fc2-4886-9161-4db321ad14e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:18:28.570994 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1108/2184] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:18:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:18:28.572105 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230d518b-58e5-4717-b98f-ceaf82474c6d service neutron] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:18:28.576261 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8066c9fb-5fc2-4886-9161-4db321ad14e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:18:28.578009 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8066c9fb-5fc2-4886-9161-4db321ad14e4 service neutron] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:18:28.581044 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230d518b-58e5-4717-b98f-ceaf82474c6d service neutron] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:18:28.586131 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16d8a65-33aa-4c46-8489-164f0c4d0841 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c4d7cde8f4ad9aa479bb09e325db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:18:28.591871 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16d8a65-33aa-4c46-8489-164f0c4d0841 service neutron] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:18:28.591871 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16d8a65-33aa-4c46-8489-164f0c4d0841 service neutron] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:18:28.591871 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16d8a65-33aa-4c46-8489-164f0c4d0841 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:18:28.592258 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1077/2185] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:18:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:18:28.600607 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16d8a65-33aa-4c46-8489-164f0c4d0841 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:18:28.603094 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16d8a65-33aa-4c46-8489-164f0c4d0841 service neutron] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:18:28.624698 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1109/2186] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:18:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:18:30.257862 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8066c9fb-5fc2-4886-9161-4db321ad14e4 service neutron] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:18:30.259966 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-8066c9fb-5fc2-4886-9161-4db321ad14e4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:18:30.281013 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8066c9fb-5fc2-4886-9161-4db321ad14e4 service neutron] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:18:30.294298 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629ed314-6ac9-49db-9ccf-6b163b526a8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:18:30.295602 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629ed314-6ac9-49db-9ccf-6b163b526a8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:18:30.297365 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629ed314-6ac9-49db-9ccf-6b163b526a8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:18:30.298567 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629ed314-6ac9-49db-9ccf-6b163b526a8f admin admin] PATH_INFO: `/v3/projects/1b37094738f44f49bb381c172488450f` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:18:30.302272 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629ed314-6ac9-49db-9ccf-6b163b526a8f admin admin] RBAC: Authorizing `identity:delete_project(project_id=1b37094738f44f49bb381c172488450f)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:18:30.306885 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629ed314-6ac9-49db-9ccf-6b163b526a8f admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:18:30.505053 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-629ed314-6ac9-49db-9ccf-6b163b526a8f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1b37094738f44f49bb381c172488450f', 'request_id': 'req-629ed314-6ac9-49db-9ccf-6b163b526a8f'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:18:30.551735 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-629ed314-6ac9-49db-9ccf-6b163b526a8f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1b37094738f44f49bb381c172488450f', 'request_id': 'req-629ed314-6ac9-49db-9ccf-6b163b526a8f'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:18:30.642516 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-629ed314-6ac9-49db-9ccf-6b163b526a8f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1b37094738f44f49bb381c172488450f', 'request_id': 'req-629ed314-6ac9-49db-9ccf-6b163b526a8f'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:18:30.688841 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-629ed314-6ac9-49db-9ccf-6b163b526a8f admin admin] The token cache is being invalidate because project 1b37094738f44f49bb381c172488450f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80468) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 18 05:18:30.690770 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-629ed314-6ac9-49db-9ccf-6b163b526a8f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:18:30.692748 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-629ed314-6ac9-49db-9ccf-6b163b526a8f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:18:30.698761 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1078/2187] 10.222.0.23 () {58 vars in 1232 bytes} [Sat Jun 18 05:18:30 2022] DELETE /identity/v3/projects/1b37094738f44f49bb381c172488450f => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:18:30.703558 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a58e33-a99f-4236-8167-6577b80d832f None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:18:30.704398 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a58e33-a99f-4236-8167-6577b80d832f None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:18:30.704850 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a58e33-a99f-4236-8167-6577b80d832f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:18:30.754104 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-66a58e33-a99f-4236-8167-6577b80d832f None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:18:30.755632 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-66a58e33-a99f-4236-8167-6577b80d832f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:18:30.914389 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1110/2188] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:18:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 18 05:18:30.923835 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ff31ad-8e51-4be3-a9c3-abad7bc6e6ec None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:18:30.924351 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ff31ad-8e51-4be3-a9c3-abad7bc6e6ec None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:18:30.924737 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ff31ad-8e51-4be3-a9c3-abad7bc6e6ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:18:30.988597 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-74ff31ad-8e51-4be3-a9c3-abad7bc6e6ec None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:18:30.989998 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-74ff31ad-8e51-4be3-a9c3-abad7bc6e6ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:18:30.999515 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1079/2189] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:18:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:18:31.005505 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a58e33-a99f-4236-8167-6577b80d832f None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:18:31.026912 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a58e33-a99f-4236-8167-6577b80d832f None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:18:31.034843 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7c9e97-72ff-4d98-9878-d95042ffb37c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:18:31.035632 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7c9e97-72ff-4d98-9878-d95042ffb37c admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:18:31.035920 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7c9e97-72ff-4d98-9878-d95042ffb37c admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:18:31.036254 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7c9e97-72ff-4d98-9878-d95042ffb37c admin admin] PATH_INFO: `/v3/domains` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:18:31.037304 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7c9e97-72ff-4d98-9878-d95042ffb37c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 05:18:31.037955 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7c9e97-72ff-4d98-9878-d95042ffb37c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:18:31.040345 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7c9e97-72ff-4d98-9878-d95042ffb37c admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:18:31.069518 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1111/2190] 10.222.0.23 () {58 vars in 1152 bytes} [Sat Jun 18 05:18:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 18 05:18:31.074360 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ff31ad-8e51-4be3-a9c3-abad7bc6e6ec None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:18:31.089322 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ff31ad-8e51-4be3-a9c3-abad7bc6e6ec None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:18:31.110819 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818378b9-4ece-4135-b15f-50a8bf2eaac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:18:31.112629 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818378b9-4ece-4135-b15f-50a8bf2eaac5 admin admin] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:18:31.120655 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818378b9-4ece-4135-b15f-50a8bf2eaac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:18:31.121046 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818378b9-4ece-4135-b15f-50a8bf2eaac5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:18:31.124144 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818378b9-4ece-4135-b15f-50a8bf2eaac5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:18:31.131303 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818378b9-4ece-4135-b15f-50a8bf2eaac5 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:18:31.185797 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7c9e97-72ff-4d98-9878-d95042ffb37c admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:18:31.187935 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-4e7c9e97-72ff-4d98-9878-d95042ffb37c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:18:31.230622 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7c9e97-72ff-4d98-9878-d95042ffb37c admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:18:31.245796 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1080/2191] 10.222.0.23 () {60 vars in 1152 bytes} [Sat Jun 18 05:18:31 2022] POST /identity/v3/projects => generated 369 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:18:31.250844 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818378b9-4ece-4135-b15f-50a8bf2eaac5 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:18:31.252481 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032fffac-b7b7-4b8f-999e-3427b0a06978 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:18:31.276257 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032fffac-b7b7-4b8f-999e-3427b0a06978 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:18:31.278996 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032fffac-b7b7-4b8f-999e-3427b0a06978 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:18:31.279992 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032fffac-b7b7-4b8f-999e-3427b0a06978 admin admin] PATH_INFO: `/v3/projects/ca9e8c9edd354806ab27e97b73cc1565` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:18:31.288482 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032fffac-b7b7-4b8f-999e-3427b0a06978 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca9e8c9edd354806ab27e97b73cc1565)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:18:31.301368 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032fffac-b7b7-4b8f-999e-3427b0a06978 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:18:31.389003 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818378b9-4ece-4135-b15f-50a8bf2eaac5 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:18:31.398152 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47b5214-7b36-4f37-a304-767bd2c1493d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:18:31.398702 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47b5214-7b36-4f37-a304-767bd2c1493d admin admin] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:18:31.398985 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47b5214-7b36-4f37-a304-767bd2c1493d admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:18:31.399260 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47b5214-7b36-4f37-a304-767bd2c1493d admin admin] PATH_INFO: `/v3/users` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:18:31.400067 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47b5214-7b36-4f37-a304-767bd2c1493d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:18:31.402312 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47b5214-7b36-4f37-a304-767bd2c1493d admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:18:31.486922 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:18:31.503425 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1081/2192] 10.222.0.23 () {60 vars in 1143 bytes} [Sat Jun 18 05:18:31 2022] POST /identity/v3/users => generated 285 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:18:31.504335 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47b5214-7b36-4f37-a304-767bd2c1493d admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:18:31.543981 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-032fffac-b7b7-4b8f-999e-3427b0a06978 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca9e8c9edd354806ab27e97b73cc1565', 'request_id': 'req-032fffac-b7b7-4b8f-999e-3427b0a06978'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:18:31.546557 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47b5214-7b36-4f37-a304-767bd2c1493d admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:18:31.570559 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-032fffac-b7b7-4b8f-999e-3427b0a06978 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca9e8c9edd354806ab27e97b73cc1565', 'request_id': 'req-032fffac-b7b7-4b8f-999e-3427b0a06978'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:18:31.579766 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-032fffac-b7b7-4b8f-999e-3427b0a06978 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca9e8c9edd354806ab27e97b73cc1565', 'request_id': 'req-032fffac-b7b7-4b8f-999e-3427b0a06978'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:18:31.590701 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:18:31.674975 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-032fffac-b7b7-4b8f-999e-3427b0a06978 admin admin] The token cache is being invalidate because project ca9e8c9edd354806ab27e97b73cc1565 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80469) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 18 05:18:31.675333 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-032fffac-b7b7-4b8f-999e-3427b0a06978 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:18:31.676565 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-032fffac-b7b7-4b8f-999e-3427b0a06978 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:18:31.680121 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1112/2193] 10.222.0.23 () {58 vars in 1232 bytes} [Sat Jun 18 05:18:31 2022] DELETE /identity/v3/projects/ca9e8c9edd354806ab27e97b73cc1565 => generated 0 bytes in 495 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:18:31.684936 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032fffac-b7b7-4b8f-999e-3427b0a06978 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:18:31.688710 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-032fffac-b7b7-4b8f-999e-3427b0a06978 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:18:31.746156 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae49ee7-536c-434b-92e1-30d3c7611a5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:18:31.749213 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae49ee7-536c-434b-92e1-30d3c7611a5e admin admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:18:31.750880 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae49ee7-536c-434b-92e1-30d3c7611a5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:18:31.751191 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae49ee7-536c-434b-92e1-30d3c7611a5e admin admin] PATH_INFO: `/v3/roles` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:18:31.756603 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae49ee7-536c-434b-92e1-30d3c7611a5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:18:31.766167 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae49ee7-536c-434b-92e1-30d3c7611a5e admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:18:31.806363 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1082/2194] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:18:31 2022] GET /identity/v3/roles => generated 1420 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:18:31.807182 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae49ee7-536c-434b-92e1-30d3c7611a5e admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:18:31.813598 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-1ae49ee7-536c-434b-92e1-30d3c7611a5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:18:31.861719 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae49ee7-536c-434b-92e1-30d3c7611a5e admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:18:31.875261 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0c6bc9-e28e-476d-b805-a8467fc1cf90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:18:31.878476 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0c6bc9-e28e-476d-b805-a8467fc1cf90 admin admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:18:31.881867 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0c6bc9-e28e-476d-b805-a8467fc1cf90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:18:31.881867 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0c6bc9-e28e-476d-b805-a8467fc1cf90 admin admin] PATH_INFO: `/v3/roles` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:18:31.881867 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0c6bc9-e28e-476d-b805-a8467fc1cf90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:18:31.883806 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0c6bc9-e28e-476d-b805-a8467fc1cf90 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:18:31.913882 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1083/2195] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:18:31 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:18:31.918335 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0c6bc9-e28e-476d-b805-a8467fc1cf90 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:18:31.919821 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032fffac-b7b7-4b8f-999e-3427b0a06978 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:18:31.946040 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0f5f8f-e3b3-4d85-a219-d15033a6fe64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:18:31.947415 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0f5f8f-e3b3-4d85-a219-d15033a6fe64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:18:31.947695 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0f5f8f-e3b3-4d85-a219-d15033a6fe64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:18:31.947969 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0f5f8f-e3b3-4d85-a219-d15033a6fe64 admin admin] PATH_INFO: `/v3/users/aa9a76f0a6be442c9773ec6646ae52e7` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:18:31.952470 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0f5f8f-e3b3-4d85-a219-d15033a6fe64 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa9a76f0a6be442c9773ec6646ae52e7)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:18:31.954650 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0f5f8f-e3b3-4d85-a219-d15033a6fe64 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:18:31.963888 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0c6bc9-e28e-476d-b805-a8467fc1cf90 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:18:31.984193 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4086ceb6-5177-410f-a3fe-9d10d599bbea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:18:31.985172 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4086ceb6-5177-410f-a3fe-9d10d599bbea admin admin] REQUEST_METHOD: `PUT` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:18:31.985493 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4086ceb6-5177-410f-a3fe-9d10d599bbea admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:18:31.985891 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4086ceb6-5177-410f-a3fe-9d10d599bbea admin admin] PATH_INFO: `/v3/projects/dceb9a79bfa74dfc923dfd2a38ee3f6f/users/db60e7b3e2d94e39a9e6de7dbbb1bdea/roles/cbf5c4c26898413d93e73b46b41046a3` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:18:32.035190 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4086ceb6-5177-410f-a3fe-9d10d599bbea admin admin] RBAC: Authorizing `identity:create_grant(project_id=dceb9a79bfa74dfc923dfd2a38ee3f6f, user_id=db60e7b3e2d94e39a9e6de7dbbb1bdea, role_id=cbf5c4c26898413d93e73b46b41046a3)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:18:32.045993 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4086ceb6-5177-410f-a3fe-9d10d599bbea admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:18:32.125484 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1084/2196] 10.222.0.23 () {60 vars in 1482 bytes} [Sat Jun 18 05:18:31 2022] PUT /identity/v3/projects/dceb9a79bfa74dfc923dfd2a38ee3f6f/users/db60e7b3e2d94e39a9e6de7dbbb1bdea/roles/cbf5c4c26898413d93e73b46b41046a3 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:18:32.132347 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91601ca1-696b-44c7-b216-6ec8f5183b44 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:18:32.133055 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91601ca1-696b-44c7-b216-6ec8f5183b44 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:18:32.134181 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91601ca1-696b-44c7-b216-6ec8f5183b44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:18:32.205145 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-91601ca1-696b-44c7-b216-6ec8f5183b44 None None] MFA Rules not processed for user `db60e7b3e2d94e39a9e6de7dbbb1bdea`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:18:32.206774 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-91601ca1-696b-44c7-b216-6ec8f5183b44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:18:32.311935 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-7a0f5f8f-e3b3-4d85-a219-d15033a6fe64 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa9a76f0a6be442c9773ec6646ae52e7'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:18:32.358450 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-7a0f5f8f-e3b3-4d85-a219-d15033a6fe64 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa9a76f0a6be442c9773ec6646ae52e7'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:18:32.359340 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-7a0f5f8f-e3b3-4d85-a219-d15033a6fe64 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa9a76f0a6be442c9773ec6646ae52e7'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:18:32.465489 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-7a0f5f8f-e3b3-4d85-a219-d15033a6fe64 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa9a76f0a6be442c9773ec6646ae52e7'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:18:32.476693 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-7a0f5f8f-e3b3-4d85-a219-d15033a6fe64 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa9a76f0a6be442c9773ec6646ae52e7'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:18:32.497223 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1085/2197] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:18:32 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 368 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 18 05:18:32.510810 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91601ca1-696b-44c7-b216-6ec8f5183b44 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:18:32.512320 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-91601ca1-696b-44c7-b216-6ec8f5183b44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:18:32.536449 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1113/2198] 10.222.0.23 () {58 vars in 1223 bytes} [Sat Jun 18 05:18:31 2022] DELETE /identity/v3/users/aa9a76f0a6be442c9773ec6646ae52e7 => generated 0 bytes in 851 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:18:32.677894 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91601ca1-696b-44c7-b216-6ec8f5183b44 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:18:32.728881 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:18:32.836069 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e4decc-f32c-4927-be09-3c0dc30a8b69 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4638e5d1fa124667b07054a87c2d353e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:18:32.837489 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e4decc-f32c-4927-be09-3c0dc30a8b69 service glance] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:18:32.837978 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e4decc-f32c-4927-be09-3c0dc30a8b69 service glance] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:18:32.839332 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e4decc-f32c-4927-be09-3c0dc30a8b69 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:18:32.902080 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae07ac1b-ddd6-460a-888a-7d1ffb224d79 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:18:32.902368 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae07ac1b-ddd6-460a-888a-7d1ffb224d79 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:18:32.905800 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae07ac1b-ddd6-460a-888a-7d1ffb224d79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:18:32.979702 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-ae07ac1b-ddd6-460a-888a-7d1ffb224d79 None None] MFA Rules not processed for user `4147695d07c24202b096fb7da07ecf37`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:18:32.983482 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-ae07ac1b-ddd6-460a-888a-7d1ffb224d79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:18:33.027187 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e4decc-f32c-4927-be09-3c0dc30a8b69 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:18:33.028899 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e4decc-f32c-4927-be09-3c0dc30a8b69 service glance] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:18:33.062225 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1086/2199] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:18:32 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 552 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 18 05:18:33.114196 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:18:33.231111 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1114/2200] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:18:32 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 334 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:18:33.244297 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e4decc-f32c-4927-be09-3c0dc30a8b69 service glance] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:18:33.258023 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-94e4decc-f32c-4927-be09-3c0dc30a8b69 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:18:33.437847 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e4decc-f32c-4927-be09-3c0dc30a8b69 service glance] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:18:33.445725 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f444300e-cbf6-4574-a74b-16aab2dfc123 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9de0428abc8400aa338aba4f7c34c94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:18:33.446860 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f444300e-cbf6-4574-a74b-16aab2dfc123 service cinder] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:18:33.447278 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f444300e-cbf6-4574-a74b-16aab2dfc123 service cinder] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:18:33.447781 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f444300e-cbf6-4574-a74b-16aab2dfc123 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:18:33.467003 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f444300e-cbf6-4574-a74b-16aab2dfc123 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:18:33.473131 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f444300e-cbf6-4574-a74b-16aab2dfc123 service cinder] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:18:33.508882 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1087/2201] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:18:33 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:18:35.090421 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae07ac1b-ddd6-460a-888a-7d1ffb224d79 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:18:35.095456 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-ae07ac1b-ddd6-460a-888a-7d1ffb224d79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:18:35.284589 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae07ac1b-ddd6-460a-888a-7d1ffb224d79 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:18:35.304623 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4efcb17-47d0-40c9-9a33-7c0e5d53a7d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:18:35.306758 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4efcb17-47d0-40c9-9a33-7c0e5d53a7d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:18:35.307053 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4efcb17-47d0-40c9-9a33-7c0e5d53a7d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:18:35.307335 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4efcb17-47d0-40c9-9a33-7c0e5d53a7d3 admin admin] PATH_INFO: `/v3/projects/a4525953a77a4f0aa54abf6510667ba9` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:18:35.309741 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4efcb17-47d0-40c9-9a33-7c0e5d53a7d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a4525953a77a4f0aa54abf6510667ba9)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:18:35.312593 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4efcb17-47d0-40c9-9a33-7c0e5d53a7d3 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:18:35.499430 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-e4efcb17-47d0-40c9-9a33-7c0e5d53a7d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4525953a77a4f0aa54abf6510667ba9', 'request_id': 'req-e4efcb17-47d0-40c9-9a33-7c0e5d53a7d3'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:18:35.518457 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-e4efcb17-47d0-40c9-9a33-7c0e5d53a7d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4525953a77a4f0aa54abf6510667ba9', 'request_id': 'req-e4efcb17-47d0-40c9-9a33-7c0e5d53a7d3'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:18:35.535825 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-e4efcb17-47d0-40c9-9a33-7c0e5d53a7d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4525953a77a4f0aa54abf6510667ba9', 'request_id': 'req-e4efcb17-47d0-40c9-9a33-7c0e5d53a7d3'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:18:35.603346 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-e4efcb17-47d0-40c9-9a33-7c0e5d53a7d3 admin admin] The token cache is being invalidate because project a4525953a77a4f0aa54abf6510667ba9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80469) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 18 05:18:35.603712 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-e4efcb17-47d0-40c9-9a33-7c0e5d53a7d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:18:35.604418 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-e4efcb17-47d0-40c9-9a33-7c0e5d53a7d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:18:35.610811 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f444300e-cbf6-4574-a74b-16aab2dfc123 service cinder] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:18:35.612599 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1115/2202] 10.222.0.23 () {58 vars in 1232 bytes} [Sat Jun 18 05:18:35 2022] DELETE /identity/v3/projects/a4525953a77a4f0aa54abf6510667ba9 => generated 0 bytes in 517 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:18:35.637322 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f444300e-cbf6-4574-a74b-16aab2dfc123 service cinder] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:18:35.798613 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc50b2f5-b968-4c6d-b7ef-372589211e15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:18:35.799713 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc50b2f5-b968-4c6d-b7ef-372589211e15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:18:35.807138 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc50b2f5-b968-4c6d-b7ef-372589211e15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:18:35.807138 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc50b2f5-b968-4c6d-b7ef-372589211e15 admin admin] PATH_INFO: `/v3/users/af1047a23f3d4fea80eb9aa42fc153c4` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:18:35.833138 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc50b2f5-b968-4c6d-b7ef-372589211e15 admin admin] RBAC: Authorizing `identity:delete_user(user_id=af1047a23f3d4fea80eb9aa42fc153c4)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:18:35.859104 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc50b2f5-b968-4c6d-b7ef-372589211e15 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:18:35.960680 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:18:35.962283 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edd8d31-02a3-45c2-a69e-7da6688ea820 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:18:35.964327 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edd8d31-02a3-45c2-a69e-7da6688ea820 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:18:35.964734 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edd8d31-02a3-45c2-a69e-7da6688ea820 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:18:36.034170 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-8edd8d31-02a3-45c2-a69e-7da6688ea820 None None] MFA Rules not processed for user `4147695d07c24202b096fb7da07ecf37`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:18:36.056077 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-8edd8d31-02a3-45c2-a69e-7da6688ea820 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:18:36.207661 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-fc50b2f5-b968-4c6d-b7ef-372589211e15 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af1047a23f3d4fea80eb9aa42fc153c4'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:18:36.208399 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-fc50b2f5-b968-4c6d-b7ef-372589211e15 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af1047a23f3d4fea80eb9aa42fc153c4'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:18:36.252351 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1116/2203] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:18:35 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:18:36.301499 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-fc50b2f5-b968-4c6d-b7ef-372589211e15 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af1047a23f3d4fea80eb9aa42fc153c4'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:18:36.329963 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-fc50b2f5-b968-4c6d-b7ef-372589211e15 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af1047a23f3d4fea80eb9aa42fc153c4'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:18:36.355283 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-fc50b2f5-b968-4c6d-b7ef-372589211e15 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af1047a23f3d4fea80eb9aa42fc153c4'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:18:36.364661 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1088/2204] 10.222.0.23 () {58 vars in 1223 bytes} [Sat Jun 18 05:18:35 2022] DELETE /identity/v3/users/af1047a23f3d4fea80eb9aa42fc153c4 => generated 0 bytes in 754 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:18:37.051333 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edd8d31-02a3-45c2-a69e-7da6688ea820 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:18:37.053912 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-8edd8d31-02a3-45c2-a69e-7da6688ea820 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:18:37.080364 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edd8d31-02a3-45c2-a69e-7da6688ea820 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:18:37.085958 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:18:37.171823 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2b1352-c36a-4753-a329-8e82b262dc1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:18:37.173139 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2b1352-c36a-4753-a329-8e82b262dc1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:18:37.173598 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2b1352-c36a-4753-a329-8e82b262dc1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:18:37.173965 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2b1352-c36a-4753-a329-8e82b262dc1f admin admin] PATH_INFO: `/v3/projects/edce75da580b457a9543339f9b001e47` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:18:37.176631 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2b1352-c36a-4753-a329-8e82b262dc1f admin admin] RBAC: Authorizing `identity:delete_project(project_id=edce75da580b457a9543339f9b001e47)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:18:37.183789 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2b1352-c36a-4753-a329-8e82b262dc1f admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:18:37.355496 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-9c2b1352-c36a-4753-a329-8e82b262dc1f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'edce75da580b457a9543339f9b001e47', 'request_id': 'req-9c2b1352-c36a-4753-a329-8e82b262dc1f'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:18:37.372978 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-9c2b1352-c36a-4753-a329-8e82b262dc1f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'edce75da580b457a9543339f9b001e47', 'request_id': 'req-9c2b1352-c36a-4753-a329-8e82b262dc1f'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:18:37.399825 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-9c2b1352-c36a-4753-a329-8e82b262dc1f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'edce75da580b457a9543339f9b001e47', 'request_id': 'req-9c2b1352-c36a-4753-a329-8e82b262dc1f'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:18:37.483304 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-9c2b1352-c36a-4753-a329-8e82b262dc1f admin admin] The token cache is being invalidate because project edce75da580b457a9543339f9b001e47 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80469) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 18 05:18:37.483712 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-9c2b1352-c36a-4753-a329-8e82b262dc1f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:18:37.486155 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-9c2b1352-c36a-4753-a329-8e82b262dc1f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:18:37.489879 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1117/2205] 10.222.0.23 () {58 vars in 1232 bytes} [Sat Jun 18 05:18:37 2022] DELETE /identity/v3/projects/edce75da580b457a9543339f9b001e47 => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:18:37.497495 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563186d7-0d72-4850-b8aa-4777d4598954 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:18:37.497495 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563186d7-0d72-4850-b8aa-4777d4598954 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:18:37.497495 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563186d7-0d72-4850-b8aa-4777d4598954 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:18:37.538518 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-563186d7-0d72-4850-b8aa-4777d4598954 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:18:37.539698 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-563186d7-0d72-4850-b8aa-4777d4598954 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:18:37.722909 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1089/2206] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:18:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:18:37.725458 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37b6d50-a526-4e33-81c1-953953af91d6 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:18:37.725763 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37b6d50-a526-4e33-81c1-953953af91d6 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:18:37.726155 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37b6d50-a526-4e33-81c1-953953af91d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:18:37.757262 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-f37b6d50-a526-4e33-81c1-953953af91d6 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:18:37.759964 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-f37b6d50-a526-4e33-81c1-953953af91d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:18:37.775008 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1118/2207] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:18:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:18:37.776228 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563186d7-0d72-4850-b8aa-4777d4598954 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:18:37.803055 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563186d7-0d72-4850-b8aa-4777d4598954 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:18:37.809117 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c741ef98-e2fa-45c3-87d0-36faaad9e38c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:18:37.809997 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c741ef98-e2fa-45c3-87d0-36faaad9e38c admin admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:18:37.810293 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c741ef98-e2fa-45c3-87d0-36faaad9e38c admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:18:37.810572 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c741ef98-e2fa-45c3-87d0-36faaad9e38c admin admin] PATH_INFO: `/v3/domains` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:18:37.811317 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c741ef98-e2fa-45c3-87d0-36faaad9e38c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 05:18:37.811633 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c741ef98-e2fa-45c3-87d0-36faaad9e38c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:18:37.813249 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c741ef98-e2fa-45c3-87d0-36faaad9e38c admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:18:37.836163 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1090/2208] 10.222.0.23 () {58 vars in 1152 bytes} [Sat Jun 18 05:18:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 05:18:37.841776 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37b6d50-a526-4e33-81c1-953953af91d6 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:18:37.854641 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37b6d50-a526-4e33-81c1-953953af91d6 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:18:37.862809 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d174de4-2592-4148-b0bc-de92cf65d077 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:18:37.863575 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d174de4-2592-4148-b0bc-de92cf65d077 admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:18:37.863919 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d174de4-2592-4148-b0bc-de92cf65d077 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:18:37.864265 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d174de4-2592-4148-b0bc-de92cf65d077 admin admin] PATH_INFO: `/v3/projects` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:18:37.865067 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d174de4-2592-4148-b0bc-de92cf65d077 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:18:37.867784 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d174de4-2592-4148-b0bc-de92cf65d077 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:18:38.015809 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c741ef98-e2fa-45c3-87d0-36faaad9e38c admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:18:38.027566 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1119/2209] 10.222.0.23 () {60 vars in 1152 bytes} [Sat Jun 18 05:18:37 2022] POST /identity/v3/projects => generated 385 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:18:38.031874 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c741ef98-e2fa-45c3-87d0-36faaad9e38c admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:18:38.135677 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71ae2b6-ac06-4f8c-a08d-0e5611dcb7fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:18:38.136412 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71ae2b6-ac06-4f8c-a08d-0e5611dcb7fe admin admin] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:18:38.136703 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71ae2b6-ac06-4f8c-a08d-0e5611dcb7fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:18:38.136966 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71ae2b6-ac06-4f8c-a08d-0e5611dcb7fe admin admin] PATH_INFO: `/v3/users` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:18:38.137869 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71ae2b6-ac06-4f8c-a08d-0e5611dcb7fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:18:38.139956 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71ae2b6-ac06-4f8c-a08d-0e5611dcb7fe admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:18:38.239897 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1091/2210] 10.222.0.23 () {60 vars in 1143 bytes} [Sat Jun 18 05:18:38 2022] POST /identity/v3/users => generated 293 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 18 05:18:38.246049 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d174de4-2592-4148-b0bc-de92cf65d077 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:18:38.264554 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d174de4-2592-4148-b0bc-de92cf65d077 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:18:38.279216 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc88a89b-4ecf-4806-8a54-ded1dd4358d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:18:38.281087 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc88a89b-4ecf-4806-8a54-ded1dd4358d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:18:38.282656 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc88a89b-4ecf-4806-8a54-ded1dd4358d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:18:38.283008 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc88a89b-4ecf-4806-8a54-ded1dd4358d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:18:38.285361 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc88a89b-4ecf-4806-8a54-ded1dd4358d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:18:38.291612 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc88a89b-4ecf-4806-8a54-ded1dd4358d7 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:18:38.340993 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1120/2211] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:18:38 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:18:38.349238 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71ae2b6-ac06-4f8c-a08d-0e5611dcb7fe admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:18:38.372931 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71ae2b6-ac06-4f8c-a08d-0e5611dcb7fe admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:18:38.398638 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6831995d-44e5-40b6-babb-3e269a2432f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:18:38.399388 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6831995d-44e5-40b6-babb-3e269a2432f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:18:38.401093 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6831995d-44e5-40b6-babb-3e269a2432f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:18:38.401410 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6831995d-44e5-40b6-babb-3e269a2432f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:18:38.405728 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6831995d-44e5-40b6-babb-3e269a2432f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:18:38.407367 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6831995d-44e5-40b6-babb-3e269a2432f6 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:18:38.435321 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1092/2212] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:18:38 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:18:38.440402 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc88a89b-4ecf-4806-8a54-ded1dd4358d7 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:18:38.457212 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc88a89b-4ecf-4806-8a54-ded1dd4358d7 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:18:38.470618 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2901bc4e-efac-4d6e-aa0a-857ae9089f68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:18:38.472155 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2901bc4e-efac-4d6e-aa0a-857ae9089f68 admin admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:18:38.472624 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2901bc4e-efac-4d6e-aa0a-857ae9089f68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:18:38.473105 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2901bc4e-efac-4d6e-aa0a-857ae9089f68 admin admin] PATH_INFO: `/v3/projects/98cc0e42bc1d4dab8cceb7801317075c/users/8f2da208d7d0444082708d790a85967c/roles/cbf5c4c26898413d93e73b46b41046a3` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:18:38.522094 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2901bc4e-efac-4d6e-aa0a-857ae9089f68 admin admin] RBAC: Authorizing `identity:create_grant(project_id=98cc0e42bc1d4dab8cceb7801317075c, user_id=8f2da208d7d0444082708d790a85967c, role_id=cbf5c4c26898413d93e73b46b41046a3)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:18:38.525981 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2901bc4e-efac-4d6e-aa0a-857ae9089f68 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:18:38.626369 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1121/2213] 10.222.0.23 () {60 vars in 1482 bytes} [Sat Jun 18 05:18:38 2022] PUT /identity/v3/projects/98cc0e42bc1d4dab8cceb7801317075c/users/8f2da208d7d0444082708d790a85967c/roles/cbf5c4c26898413d93e73b46b41046a3 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:18:38.628113 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b77f5a-4f62-40c0-b4da-2a1b3454ab81 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:18:38.629112 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b77f5a-4f62-40c0-b4da-2a1b3454ab81 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:18:38.629566 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b77f5a-4f62-40c0-b4da-2a1b3454ab81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:18:38.650322 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:18:38.670714 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-73b77f5a-4f62-40c0-b4da-2a1b3454ab81 None None] MFA Rules not processed for user `8f2da208d7d0444082708d790a85967c`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:18:38.671917 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-73b77f5a-4f62-40c0-b4da-2a1b3454ab81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:18:38.760487 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1093/2214] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:18:38 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 18 05:18:38.763770 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2901bc4e-efac-4d6e-aa0a-857ae9089f68 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:18:38.859638 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:18:38.894685 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2901bc4e-efac-4d6e-aa0a-857ae9089f68 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:18:38.909076 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97268070-e92a-4448-8b2b-b65f775df0df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:18:38.911043 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97268070-e92a-4448-8b2b-b65f775df0df admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:18:38.911406 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97268070-e92a-4448-8b2b-b65f775df0df admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:18:38.911723 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97268070-e92a-4448-8b2b-b65f775df0df admin admin] PATH_INFO: `/v3/projects` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:18:38.912453 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97268070-e92a-4448-8b2b-b65f775df0df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:18:38.915807 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97268070-e92a-4448-8b2b-b65f775df0df admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:18:39.055009 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1122/2215] 10.222.0.23 () {60 vars in 1152 bytes} [Sat Jun 18 05:18:38 2022] POST /identity/v3/projects => generated 383 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:18:39.064026 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b77f5a-4f62-40c0-b4da-2a1b3454ab81 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:18:39.094623 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b77f5a-4f62-40c0-b4da-2a1b3454ab81 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:18:39.243007 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a196b41f-c876-44cc-b5b7-bf0173656594 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:18:39.244312 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a196b41f-c876-44cc-b5b7-bf0173656594 admin admin] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:18:39.244915 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a196b41f-c876-44cc-b5b7-bf0173656594 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:18:39.245730 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a196b41f-c876-44cc-b5b7-bf0173656594 admin admin] PATH_INFO: `/v3/users` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:18:39.246611 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a196b41f-c876-44cc-b5b7-bf0173656594 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:18:39.249884 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a196b41f-c876-44cc-b5b7-bf0173656594 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:18:39.397682 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1094/2216] 10.222.0.23 () {60 vars in 1143 bytes} [Sat Jun 18 05:18:39 2022] POST /identity/v3/users => generated 292 bytes in 334 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:18:39.403109 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97268070-e92a-4448-8b2b-b65f775df0df admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:18:39.417373 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97268070-e92a-4448-8b2b-b65f775df0df admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:18:39.432824 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482d88fb-dce4-4dd7-8520-a71b4f0822f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:18:39.433669 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482d88fb-dce4-4dd7-8520-a71b4f0822f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:18:39.433954 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482d88fb-dce4-4dd7-8520-a71b4f0822f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:18:39.434230 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482d88fb-dce4-4dd7-8520-a71b4f0822f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:18:39.435035 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482d88fb-dce4-4dd7-8520-a71b4f0822f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:18:39.436668 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482d88fb-dce4-4dd7-8520-a71b4f0822f6 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:18:39.452724 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1123/2217] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:18:39 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:18:39.454915 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a196b41f-c876-44cc-b5b7-bf0173656594 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:18:39.466607 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a196b41f-c876-44cc-b5b7-bf0173656594 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:18:39.471732 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a71d489-9212-44c6-b0f4-21357a596841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:18:39.472596 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a71d489-9212-44c6-b0f4-21357a596841 admin admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:18:39.473291 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a71d489-9212-44c6-b0f4-21357a596841 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:18:39.473429 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a71d489-9212-44c6-b0f4-21357a596841 admin admin] PATH_INFO: `/v3/roles` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:18:39.474351 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a71d489-9212-44c6-b0f4-21357a596841 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:18:39.476136 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a71d489-9212-44c6-b0f4-21357a596841 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:18:39.490250 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1095/2218] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:18:39 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:18:39.494817 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482d88fb-dce4-4dd7-8520-a71b4f0822f6 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:18:39.508208 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482d88fb-dce4-4dd7-8520-a71b4f0822f6 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:18:39.515751 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b535f7-eca3-4d77-bd1b-533f9be1bad3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:18:39.516824 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b535f7-eca3-4d77-bd1b-533f9be1bad3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:18:39.517238 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b535f7-eca3-4d77-bd1b-533f9be1bad3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:18:39.517648 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b535f7-eca3-4d77-bd1b-533f9be1bad3 admin admin] PATH_INFO: `/v3/projects/634836846aa84596aa575099457455a5/users/dcf12760393546e3ace39b58b2e425bf/roles/cbf5c4c26898413d93e73b46b41046a3` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:18:39.541481 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b535f7-eca3-4d77-bd1b-533f9be1bad3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=634836846aa84596aa575099457455a5, user_id=dcf12760393546e3ace39b58b2e425bf, role_id=cbf5c4c26898413d93e73b46b41046a3)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:18:39.546173 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b535f7-eca3-4d77-bd1b-533f9be1bad3 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:18:39.652180 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1124/2219] 10.222.0.23 () {60 vars in 1482 bytes} [Sat Jun 18 05:18:39 2022] PUT /identity/v3/projects/634836846aa84596aa575099457455a5/users/dcf12760393546e3ace39b58b2e425bf/roles/cbf5c4c26898413d93e73b46b41046a3 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 18 05:18:39.667545 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1f8bd9-f480-4043-92d7-e3499b38c3af None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:18:39.668559 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1f8bd9-f480-4043-92d7-e3499b38c3af None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:18:39.668848 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1f8bd9-f480-4043-92d7-e3499b38c3af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:18:39.715624 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-7e1f8bd9-f480-4043-92d7-e3499b38c3af None None] MFA Rules not processed for user `dcf12760393546e3ace39b58b2e425bf`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:18:39.718616 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-7e1f8bd9-f480-4043-92d7-e3499b38c3af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:18:39.819588 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1096/2220] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:18:39 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:18:39.824359 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b535f7-eca3-4d77-bd1b-533f9be1bad3 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:18:39.939350 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b535f7-eca3-4d77-bd1b-533f9be1bad3 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:18:39.945456 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051a2cf9-2f93-456c-a7fc-a7077a55f199 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:18:39.946307 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051a2cf9-2f93-456c-a7fc-a7077a55f199 admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:18:39.947356 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051a2cf9-2f93-456c-a7fc-a7077a55f199 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:18:39.950681 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051a2cf9-2f93-456c-a7fc-a7077a55f199 admin admin] PATH_INFO: `/v3/projects` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:18:39.950681 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051a2cf9-2f93-456c-a7fc-a7077a55f199 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:18:39.950681 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051a2cf9-2f93-456c-a7fc-a7077a55f199 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:18:40.070822 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1125/2221] 10.222.0.23 () {60 vars in 1152 bytes} [Sat Jun 18 05:18:39 2022] POST /identity/v3/projects => generated 385 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:18:40.072602 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1f8bd9-f480-4043-92d7-e3499b38c3af None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:18:40.080140 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:18:40.090492 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1f8bd9-f480-4043-92d7-e3499b38c3af None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:18:40.267703 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc49e12e-5656-4b92-bc82-5b74ead693e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:18:40.267937 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc49e12e-5656-4b92-bc82-5b74ead693e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:18:40.268105 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc49e12e-5656-4b92-bc82-5b74ead693e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:18:40.268397 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc49e12e-5656-4b92-bc82-5b74ead693e5 admin admin] PATH_INFO: `/v3/users` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:18:40.269124 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc49e12e-5656-4b92-bc82-5b74ead693e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:18:40.278212 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc49e12e-5656-4b92-bc82-5b74ead693e5 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:18:40.366104 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1097/2222] 10.222.0.23 () {60 vars in 1143 bytes} [Sat Jun 18 05:18:40 2022] POST /identity/v3/users => generated 299 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 18 05:18:40.370794 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051a2cf9-2f93-456c-a7fc-a7077a55f199 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:18:40.385463 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051a2cf9-2f93-456c-a7fc-a7077a55f199 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:18:40.402678 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d011a187-1d80-4032-b91a-c02f72faa3f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:18:40.403644 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d011a187-1d80-4032-b91a-c02f72faa3f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:18:40.404169 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d011a187-1d80-4032-b91a-c02f72faa3f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:18:40.404602 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d011a187-1d80-4032-b91a-c02f72faa3f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:18:40.414128 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d011a187-1d80-4032-b91a-c02f72faa3f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:18:40.414128 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d011a187-1d80-4032-b91a-c02f72faa3f2 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:18:40.420576 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1126/2223] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:18:40 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:18:40.424683 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc49e12e-5656-4b92-bc82-5b74ead693e5 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:18:40.447094 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc49e12e-5656-4b92-bc82-5b74ead693e5 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:18:40.447094 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5d5ab5-c28a-47dd-ac01-03fad7a9375f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:18:40.447094 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5d5ab5-c28a-47dd-ac01-03fad7a9375f admin admin] REQUEST_METHOD: `PUT` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:18:40.447094 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5d5ab5-c28a-47dd-ac01-03fad7a9375f admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:18:40.448484 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5d5ab5-c28a-47dd-ac01-03fad7a9375f admin admin] PATH_INFO: `/v3/projects/e18c790eebf444f1bc752d1c719a819b/users/df2c14232ce44b3c8ac6152993422602/roles/21c65f2c51bd4dd283d13507d2714467` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:18:40.488940 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5d5ab5-c28a-47dd-ac01-03fad7a9375f admin admin] RBAC: Authorizing `identity:create_grant(project_id=e18c790eebf444f1bc752d1c719a819b, user_id=df2c14232ce44b3c8ac6152993422602, role_id=21c65f2c51bd4dd283d13507d2714467)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:18:40.492253 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5d5ab5-c28a-47dd-ac01-03fad7a9375f admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:18:40.597892 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1098/2224] 10.222.0.23 () {60 vars in 1482 bytes} [Sat Jun 18 05:18:40 2022] PUT /identity/v3/projects/e18c790eebf444f1bc752d1c719a819b/users/df2c14232ce44b3c8ac6152993422602/roles/21c65f2c51bd4dd283d13507d2714467 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:18:40.601998 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d011a187-1d80-4032-b91a-c02f72faa3f2 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:18:40.616844 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d011a187-1d80-4032-b91a-c02f72faa3f2 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:18:40.760071 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7e6e8a-9cf2-4741-b819-20203fe94e21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:18:40.766758 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7e6e8a-9cf2-4741-b819-20203fe94e21 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:18:40.766758 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7e6e8a-9cf2-4741-b819-20203fe94e21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:18:40.766758 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7e6e8a-9cf2-4741-b819-20203fe94e21 admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:18:40.766758 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7e6e8a-9cf2-4741-b819-20203fe94e21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:18:40.766758 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7e6e8a-9cf2-4741-b819-20203fe94e21 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:18:40.766758 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:18:40.805942 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1127/2225] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:18:40 2022] GET /identity/v3/roles => generated 1420 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:18:40.810596 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5d5ab5-c28a-47dd-ac01-03fad7a9375f admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:18:40.823532 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5d5ab5-c28a-47dd-ac01-03fad7a9375f admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:18:40.840809 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169627dc-98a2-4f75-ba51-e45b4e5246b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:18:40.841357 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169627dc-98a2-4f75-ba51-e45b4e5246b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:18:40.844302 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169627dc-98a2-4f75-ba51-e45b4e5246b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:18:40.844514 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169627dc-98a2-4f75-ba51-e45b4e5246b4 admin admin] PATH_INFO: `/v3/projects/e18c790eebf444f1bc752d1c719a819b/users/df2c14232ce44b3c8ac6152993422602/roles/21c65f2c51bd4dd283d13507d2714467` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:18:40.850662 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169627dc-98a2-4f75-ba51-e45b4e5246b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e18c790eebf444f1bc752d1c719a819b, user_id=df2c14232ce44b3c8ac6152993422602, role_id=21c65f2c51bd4dd283d13507d2714467)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:18:40.856750 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169627dc-98a2-4f75-ba51-e45b4e5246b4 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:18:40.871828 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1099/2226] 10.222.0.23 () {60 vars in 1482 bytes} [Sat Jun 18 05:18:40 2022] PUT /identity/v3/projects/e18c790eebf444f1bc752d1c719a819b/users/df2c14232ce44b3c8ac6152993422602/roles/21c65f2c51bd4dd283d13507d2714467 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:18:40.883037 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2b3c6d-ad9e-406f-a5cd-1d734e8546e6 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:18:40.883447 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2b3c6d-ad9e-406f-a5cd-1d734e8546e6 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:18:40.884708 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2b3c6d-ad9e-406f-a5cd-1d734e8546e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:18:40.990315 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-7d2b3c6d-ad9e-406f-a5cd-1d734e8546e6 None None] MFA Rules not processed for user `df2c14232ce44b3c8ac6152993422602`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:18:40.990315 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-7d2b3c6d-ad9e-406f-a5cd-1d734e8546e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:18:41.179222 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1128/2227] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:18:40 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:18:41.195411 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169627dc-98a2-4f75-ba51-e45b4e5246b4 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:18:41.216522 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-169627dc-98a2-4f75-ba51-e45b4e5246b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:18:41.334682 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169627dc-98a2-4f75-ba51-e45b4e5246b4 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:18:41.338809 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923d87a9-6234-4fa5-90d6-a76561a992ef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9de0428abc8400aa338aba4f7c34c94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:18:41.339583 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923d87a9-6234-4fa5-90d6-a76561a992ef service cinder] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:18:41.339929 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923d87a9-6234-4fa5-90d6-a76561a992ef service cinder] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:18:41.340272 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923d87a9-6234-4fa5-90d6-a76561a992ef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:18:41.612534 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923d87a9-6234-4fa5-90d6-a76561a992ef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:18:41.612534 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923d87a9-6234-4fa5-90d6-a76561a992ef service cinder] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:18:41.612534 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1100/2228] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:18:41 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 18 05:18:42.807978 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eef9d4-6e49-4ada-a1d7-28e7d2f62923 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:18:42.808559 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eef9d4-6e49-4ada-a1d7-28e7d2f62923 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:18:42.808661 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eef9d4-6e49-4ada-a1d7-28e7d2f62923 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:18:42.847477 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-16eef9d4-6e49-4ada-a1d7-28e7d2f62923 None None] MFA Rules not processed for user `2d5c6fe0cb7e4da69cb69d8de910a82d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:18:42.849101 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-16eef9d4-6e49-4ada-a1d7-28e7d2f62923 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:18:42.952848 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1129/2229] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:18:42 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 18 05:18:43.213330 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923d87a9-6234-4fa5-90d6-a76561a992ef service cinder] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:18:43.215538 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-923d87a9-6234-4fa5-90d6-a76561a992ef service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:18:43.234310 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:18:43.334913 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923d87a9-6234-4fa5-90d6-a76561a992ef service cinder] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:18:43.341625 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e4ecaf-07dc-4d6e-af65-d083369b7613 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15e7f45ee0744c1d9602187eac8e8b80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:18:43.342503 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e4ecaf-07dc-4d6e-af65-d083369b7613 service nova] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:18:43.342956 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e4ecaf-07dc-4d6e-af65-d083369b7613 service nova] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:18:43.343698 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e4ecaf-07dc-4d6e-af65-d083369b7613 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:18:43.345831 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-d3e4ecaf-07dc-4d6e-af65-d083369b7613 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:18:43.462853 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e4ecaf-07dc-4d6e-af65-d083369b7613 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:18:43.464418 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e4ecaf-07dc-4d6e-af65-d083369b7613 service nova] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:18:43.480151 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1101/2230] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:18:43 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:18:43.986151 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16eef9d4-6e49-4ada-a1d7-28e7d2f62923 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:18:44.001225 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16eef9d4-6e49-4ada-a1d7-28e7d2f62923 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:18:44.016822 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e87305-957e-4cc2-8497-a919b93f95f8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9de0428abc8400aa338aba4f7c34c94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:18:44.017541 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e87305-957e-4cc2-8497-a919b93f95f8 service cinder] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:18:44.017830 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e87305-957e-4cc2-8497-a919b93f95f8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:18:44.018104 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e87305-957e-4cc2-8497-a919b93f95f8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:18:44.089128 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e87305-957e-4cc2-8497-a919b93f95f8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:18:44.090670 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e87305-957e-4cc2-8497-a919b93f95f8 service cinder] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:18:44.101686 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1130/2231] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:18:43 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:18:45.773614 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e4ecaf-07dc-4d6e-af65-d083369b7613 service nova] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:18:45.775767 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-d3e4ecaf-07dc-4d6e-af65-d083369b7613 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:18:45.793877 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e4ecaf-07dc-4d6e-af65-d083369b7613 service nova] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:18:45.797961 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377f43c0-0d23-45a6-b23c-5750ec796978 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9de0428abc8400aa338aba4f7c34c94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:18:45.798614 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377f43c0-0d23-45a6-b23c-5750ec796978 service cinder] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:18:45.799084 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377f43c0-0d23-45a6-b23c-5750ec796978 service cinder] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:18:45.799768 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377f43c0-0d23-45a6-b23c-5750ec796978 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:18:45.811749 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377f43c0-0d23-45a6-b23c-5750ec796978 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:18:45.813480 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377f43c0-0d23-45a6-b23c-5750ec796978 service cinder] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:18:45.826750 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1102/2232] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:18:45 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:18:46.443660 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0263f8-ffa9-4773-b3d3-1757db859fdb None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:18:46.444546 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0263f8-ffa9-4773-b3d3-1757db859fdb None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:18:46.444832 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0263f8-ffa9-4773-b3d3-1757db859fdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:18:46.454073 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:18:46.465542 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-4e0263f8-ffa9-4773-b3d3-1757db859fdb None None] MFA Rules not processed for user `55bc735165cd4029bd5d581fd0187f9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:18:46.466772 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-4e0263f8-ffa9-4773-b3d3-1757db859fdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:18:46.475430 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1131/2233] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:18:46 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:18:47.254669 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba67c900-3dd6-4adf-9c3b-43884958d2be None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:18:47.255584 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba67c900-3dd6-4adf-9c3b-43884958d2be None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:18:47.256061 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba67c900-3dd6-4adf-9c3b-43884958d2be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:18:47.279812 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-ba67c900-3dd6-4adf-9c3b-43884958d2be None None] MFA Rules not processed for user `e817fafd0c1b4bf9944481cf116f1260`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:18:47.281425 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-ba67c900-3dd6-4adf-9c3b-43884958d2be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:18:47.435438 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1103/2234] 10.222.0.23 () {58 vars in 1009 bytes} [Sat Jun 18 05:18:47 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:18:47.453266 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0263f8-ffa9-4773-b3d3-1757db859fdb None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:18:47.454282 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-4e0263f8-ffa9-4773-b3d3-1757db859fdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:18:47.591479 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0263f8-ffa9-4773-b3d3-1757db859fdb None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:18:47.599666 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b93ef7-f48f-420d-b494-a35299bf94c3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16b49e866398400580ae48aa5a96fc04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:18:47.600622 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b93ef7-f48f-420d-b494-a35299bf94c3 service swift] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:18:47.602399 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b93ef7-f48f-420d-b494-a35299bf94c3 service swift] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:18:47.602874 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b93ef7-f48f-420d-b494-a35299bf94c3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:18:47.624430 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b93ef7-f48f-420d-b494-a35299bf94c3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:18:47.626516 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b93ef7-f48f-420d-b494-a35299bf94c3 service swift] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:18:47.668541 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1132/2235] 10.222.0.23 () {60 vars in 1384 bytes} [Sat Jun 18 05:18:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 18 05:18:52.090359 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba67c900-3dd6-4adf-9c3b-43884958d2be None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:18:52.148382 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:18:52.174868 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba67c900-3dd6-4adf-9c3b-43884958d2be None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:18:52.181479 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6bf7ed-c0d5-4e65-81b1-d028f6e39e5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:18:52.185019 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6bf7ed-c0d5-4e65-81b1-d028f6e39e5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:18:52.185259 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6bf7ed-c0d5-4e65-81b1-d028f6e39e5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:18:52.185587 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6bf7ed-c0d5-4e65-81b1-d028f6e39e5c admin admin] PATH_INFO: `/v3/users/8f2da208d7d0444082708d790a85967c` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:18:52.186832 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6bf7ed-c0d5-4e65-81b1-d028f6e39e5c admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f2da208d7d0444082708d790a85967c)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:18:52.189203 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6bf7ed-c0d5-4e65-81b1-d028f6e39e5c admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:18:52.542336 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-6d6bf7ed-c0d5-4e65-81b1-d028f6e39e5c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f2da208d7d0444082708d790a85967c'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:18:52.543399 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-6d6bf7ed-c0d5-4e65-81b1-d028f6e39e5c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f2da208d7d0444082708d790a85967c'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:18:52.643458 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-6d6bf7ed-c0d5-4e65-81b1-d028f6e39e5c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f2da208d7d0444082708d790a85967c'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:18:52.669187 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-6d6bf7ed-c0d5-4e65-81b1-d028f6e39e5c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f2da208d7d0444082708d790a85967c'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:18:52.686238 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-6d6bf7ed-c0d5-4e65-81b1-d028f6e39e5c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f2da208d7d0444082708d790a85967c'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:18:52.696845 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1104/2236] 10.222.0.23 () {58 vars in 1223 bytes} [Sat Jun 18 05:18:52 2022] DELETE /identity/v3/users/8f2da208d7d0444082708d790a85967c => generated 0 bytes in 607 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:18:52.701581 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60131c3-2c49-424b-856b-c401a2754fa1 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:18:52.702172 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60131c3-2c49-424b-856b-c401a2754fa1 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:18:52.702172 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60131c3-2c49-424b-856b-c401a2754fa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:18:52.735482 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-a60131c3-2c49-424b-856b-c401a2754fa1 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:18:52.736757 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-a60131c3-2c49-424b-856b-c401a2754fa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:18:52.843203 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1133/2237] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:18:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:18:52.856441 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6bf7ed-c0d5-4e65-81b1-d028f6e39e5c admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:18:52.858494 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-6d6bf7ed-c0d5-4e65-81b1-d028f6e39e5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:18:52.964366 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6bf7ed-c0d5-4e65-81b1-d028f6e39e5c admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:18:52.968911 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65eab934-2e26-4061-8ff1-5bf4e6dff774 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c4d7cde8f4ad9aa479bb09e325db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:18:52.969888 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eab934-2e26-4061-8ff1-5bf4e6dff774 service neutron] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:18:52.970180 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eab934-2e26-4061-8ff1-5bf4e6dff774 service neutron] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:18:52.970461 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eab934-2e26-4061-8ff1-5bf4e6dff774 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:18:52.999194 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65eab934-2e26-4061-8ff1-5bf4e6dff774 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:18:53.000852 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65eab934-2e26-4061-8ff1-5bf4e6dff774 service neutron] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:18:53.021636 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1105/2238] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:18:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:18:53.692233 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60131c3-2c49-424b-856b-c401a2754fa1 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:18:53.692233 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-a60131c3-2c49-424b-856b-c401a2754fa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:18:53.708163 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60131c3-2c49-424b-856b-c401a2754fa1 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:18:53.732311 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a02da5e-d751-449f-82e2-983459f3169f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:18:53.734756 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a02da5e-d751-449f-82e2-983459f3169f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:18:53.735409 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a02da5e-d751-449f-82e2-983459f3169f admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:18:53.737653 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a02da5e-d751-449f-82e2-983459f3169f admin admin] PATH_INFO: `/v3/projects/98cc0e42bc1d4dab8cceb7801317075c` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:18:53.740509 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a02da5e-d751-449f-82e2-983459f3169f admin admin] RBAC: Authorizing `identity:delete_project(project_id=98cc0e42bc1d4dab8cceb7801317075c)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:18:53.745533 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a02da5e-d751-449f-82e2-983459f3169f admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:18:53.960352 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-9a02da5e-d751-449f-82e2-983459f3169f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '98cc0e42bc1d4dab8cceb7801317075c', 'request_id': 'req-9a02da5e-d751-449f-82e2-983459f3169f'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:18:53.971135 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-9a02da5e-d751-449f-82e2-983459f3169f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '98cc0e42bc1d4dab8cceb7801317075c', 'request_id': 'req-9a02da5e-d751-449f-82e2-983459f3169f'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:18:53.995822 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-9a02da5e-d751-449f-82e2-983459f3169f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '98cc0e42bc1d4dab8cceb7801317075c', 'request_id': 'req-9a02da5e-d751-449f-82e2-983459f3169f'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:18:54.067367 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-9a02da5e-d751-449f-82e2-983459f3169f admin admin] The token cache is being invalidate because project 98cc0e42bc1d4dab8cceb7801317075c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80469) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 18 05:18:54.067964 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-9a02da5e-d751-449f-82e2-983459f3169f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:18:54.069424 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-9a02da5e-d751-449f-82e2-983459f3169f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:18:54.072109 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1134/2239] 10.222.0.23 () {58 vars in 1232 bytes} [Sat Jun 18 05:18:53 2022] DELETE /identity/v3/projects/98cc0e42bc1d4dab8cceb7801317075c => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:18:54.077917 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65eab934-2e26-4061-8ff1-5bf4e6dff774 service neutron] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:18:54.091342 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65eab934-2e26-4061-8ff1-5bf4e6dff774 service neutron] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:18:54.186611 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d95b58-b1d7-4105-aafc-1b2cf175eab4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:18:54.187632 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d95b58-b1d7-4105-aafc-1b2cf175eab4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:18:54.187983 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d95b58-b1d7-4105-aafc-1b2cf175eab4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:18:54.188317 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d95b58-b1d7-4105-aafc-1b2cf175eab4 admin admin] PATH_INFO: `/v3/users/dcf12760393546e3ace39b58b2e425bf` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:18:54.189551 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d95b58-b1d7-4105-aafc-1b2cf175eab4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dcf12760393546e3ace39b58b2e425bf)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:18:54.191458 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d95b58-b1d7-4105-aafc-1b2cf175eab4 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:18:54.194244 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:18:54.488898 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-54d95b58-b1d7-4105-aafc-1b2cf175eab4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dcf12760393546e3ace39b58b2e425bf'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:18:54.489811 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-54d95b58-b1d7-4105-aafc-1b2cf175eab4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dcf12760393546e3ace39b58b2e425bf'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:18:54.579980 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-54d95b58-b1d7-4105-aafc-1b2cf175eab4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dcf12760393546e3ace39b58b2e425bf'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:18:54.600675 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-54d95b58-b1d7-4105-aafc-1b2cf175eab4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dcf12760393546e3ace39b58b2e425bf'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:18:54.622888 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-54d95b58-b1d7-4105-aafc-1b2cf175eab4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dcf12760393546e3ace39b58b2e425bf'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:18:54.633164 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1106/2240] 10.222.0.23 () {58 vars in 1223 bytes} [Sat Jun 18 05:18:54 2022] DELETE /identity/v3/users/dcf12760393546e3ace39b58b2e425bf => generated 0 bytes in 553 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:18:55.389092 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a02da5e-d751-449f-82e2-983459f3169f admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:18:55.389092 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-9a02da5e-d751-449f-82e2-983459f3169f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:18:55.455422 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a02da5e-d751-449f-82e2-983459f3169f admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:18:55.482296 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:18:55.636893 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb1b8eb-21a9-4c5e-b9fc-39974bf3fdcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:18:55.638257 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb1b8eb-21a9-4c5e-b9fc-39974bf3fdcb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:18:55.638815 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb1b8eb-21a9-4c5e-b9fc-39974bf3fdcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:18:55.639313 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb1b8eb-21a9-4c5e-b9fc-39974bf3fdcb admin admin] PATH_INFO: `/v3/projects/634836846aa84596aa575099457455a5` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:18:55.641832 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb1b8eb-21a9-4c5e-b9fc-39974bf3fdcb admin admin] RBAC: Authorizing `identity:delete_project(project_id=634836846aa84596aa575099457455a5)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:18:55.646394 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb1b8eb-21a9-4c5e-b9fc-39974bf3fdcb admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:18:55.789754 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-feb1b8eb-21a9-4c5e-b9fc-39974bf3fdcb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '634836846aa84596aa575099457455a5', 'request_id': 'req-feb1b8eb-21a9-4c5e-b9fc-39974bf3fdcb'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:18:55.797012 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-feb1b8eb-21a9-4c5e-b9fc-39974bf3fdcb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '634836846aa84596aa575099457455a5', 'request_id': 'req-feb1b8eb-21a9-4c5e-b9fc-39974bf3fdcb'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:18:55.805455 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-feb1b8eb-21a9-4c5e-b9fc-39974bf3fdcb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '634836846aa84596aa575099457455a5', 'request_id': 'req-feb1b8eb-21a9-4c5e-b9fc-39974bf3fdcb'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:18:55.996761 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-feb1b8eb-21a9-4c5e-b9fc-39974bf3fdcb admin admin] The token cache is being invalidate because project 634836846aa84596aa575099457455a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80469) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 18 05:18:55.996761 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-feb1b8eb-21a9-4c5e-b9fc-39974bf3fdcb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:18:56.021375 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-feb1b8eb-21a9-4c5e-b9fc-39974bf3fdcb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:18:56.053973 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1135/2241] 10.222.0.23 () {58 vars in 1232 bytes} [Sat Jun 18 05:18:55 2022] DELETE /identity/v3/projects/634836846aa84596aa575099457455a5 => generated 0 bytes in 653 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:18:56.054922 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d95b58-b1d7-4105-aafc-1b2cf175eab4 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:18:56.067727 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-54d95b58-b1d7-4105-aafc-1b2cf175eab4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:18:56.098981 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d95b58-b1d7-4105-aafc-1b2cf175eab4 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:18:56.264534 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bc1eea-e379-47a0-bfa4-85956ce4294b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:18:56.265458 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bc1eea-e379-47a0-bfa4-85956ce4294b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:18:56.274321 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bc1eea-e379-47a0-bfa4-85956ce4294b admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:18:56.274698 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bc1eea-e379-47a0-bfa4-85956ce4294b admin admin] PATH_INFO: `/v3/users/df2c14232ce44b3c8ac6152993422602` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:18:56.276092 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bc1eea-e379-47a0-bfa4-85956ce4294b admin admin] RBAC: Authorizing `identity:delete_user(user_id=df2c14232ce44b3c8ac6152993422602)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:18:56.278526 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bc1eea-e379-47a0-bfa4-85956ce4294b admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:18:56.717945 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-f5bc1eea-e379-47a0-bfa4-85956ce4294b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df2c14232ce44b3c8ac6152993422602'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:18:56.718670 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-f5bc1eea-e379-47a0-bfa4-85956ce4294b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df2c14232ce44b3c8ac6152993422602'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:18:56.814008 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-f5bc1eea-e379-47a0-bfa4-85956ce4294b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df2c14232ce44b3c8ac6152993422602'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:18:56.830216 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-f5bc1eea-e379-47a0-bfa4-85956ce4294b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df2c14232ce44b3c8ac6152993422602'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:18:56.838648 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-f5bc1eea-e379-47a0-bfa4-85956ce4294b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df2c14232ce44b3c8ac6152993422602'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:18:56.841247 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:18:56.851703 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1107/2242] 10.222.0.23 () {58 vars in 1223 bytes} [Sat Jun 18 05:18:56 2022] DELETE /identity/v3/users/df2c14232ce44b3c8ac6152993422602 => generated 0 bytes in 794 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:18:57.548812 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb1b8eb-21a9-4c5e-b9fc-39974bf3fdcb admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:18:57.551177 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-feb1b8eb-21a9-4c5e-b9fc-39974bf3fdcb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:18:57.582555 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb1b8eb-21a9-4c5e-b9fc-39974bf3fdcb admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:18:57.722335 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be46f9c-0351-43fa-a7ca-9de2d86a85ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:18:57.732285 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be46f9c-0351-43fa-a7ca-9de2d86a85ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:18:57.732285 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be46f9c-0351-43fa-a7ca-9de2d86a85ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:18:57.732285 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be46f9c-0351-43fa-a7ca-9de2d86a85ab admin admin] PATH_INFO: `/v3/projects/e18c790eebf444f1bc752d1c719a819b` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:18:57.734390 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be46f9c-0351-43fa-a7ca-9de2d86a85ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=e18c790eebf444f1bc752d1c719a819b)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:18:57.735981 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be46f9c-0351-43fa-a7ca-9de2d86a85ab admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:18:57.925081 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:18:57.937307 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-1be46f9c-0351-43fa-a7ca-9de2d86a85ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e18c790eebf444f1bc752d1c719a819b', 'request_id': 'req-1be46f9c-0351-43fa-a7ca-9de2d86a85ab'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:18:57.945154 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-1be46f9c-0351-43fa-a7ca-9de2d86a85ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e18c790eebf444f1bc752d1c719a819b', 'request_id': 'req-1be46f9c-0351-43fa-a7ca-9de2d86a85ab'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:18:57.956762 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-1be46f9c-0351-43fa-a7ca-9de2d86a85ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e18c790eebf444f1bc752d1c719a819b', 'request_id': 'req-1be46f9c-0351-43fa-a7ca-9de2d86a85ab'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:18:58.036921 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-1be46f9c-0351-43fa-a7ca-9de2d86a85ab admin admin] The token cache is being invalidate because project e18c790eebf444f1bc752d1c719a819b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80469) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 18 05:18:58.037210 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-1be46f9c-0351-43fa-a7ca-9de2d86a85ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:18:58.037991 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-1be46f9c-0351-43fa-a7ca-9de2d86a85ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:18:58.040010 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1136/2243] 10.222.0.23 () {58 vars in 1232 bytes} [Sat Jun 18 05:18:57 2022] DELETE /identity/v3/projects/e18c790eebf444f1bc752d1c719a819b => generated 0 bytes in 491 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:18:58.046408 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08069842-6586-4e01-88a0-0f6ca5d9ee1d None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:18:58.047644 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08069842-6586-4e01-88a0-0f6ca5d9ee1d None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:18:58.048127 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08069842-6586-4e01-88a0-0f6ca5d9ee1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:18:58.122115 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-08069842-6586-4e01-88a0-0f6ca5d9ee1d None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:18:58.124704 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-08069842-6586-4e01-88a0-0f6ca5d9ee1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:18:58.448190 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1108/2244] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:18:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 403 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:18:58.454787 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf36dbf-1f82-4f83-ab15-681047c38bc5 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:18:58.455293 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf36dbf-1f82-4f83-ab15-681047c38bc5 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:18:58.455761 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf36dbf-1f82-4f83-ab15-681047c38bc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:18:58.535630 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-abf36dbf-1f82-4f83-ab15-681047c38bc5 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:18:58.538090 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-abf36dbf-1f82-4f83-ab15-681047c38bc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:18:58.571337 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08069842-6586-4e01-88a0-0f6ca5d9ee1d None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:18:58.577252 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1137/2245] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:18:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:18:58.583359 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08069842-6586-4e01-88a0-0f6ca5d9ee1d None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:18:58.602311 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7953d36e-5385-4ad0-ac53-c68129d2758f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:18:58.604270 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7953d36e-5385-4ad0-ac53-c68129d2758f admin admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:18:58.604887 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7953d36e-5385-4ad0-ac53-c68129d2758f admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:18:58.605257 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7953d36e-5385-4ad0-ac53-c68129d2758f admin admin] PATH_INFO: `/v3/domains` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:18:58.606183 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7953d36e-5385-4ad0-ac53-c68129d2758f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 05:18:58.606727 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7953d36e-5385-4ad0-ac53-c68129d2758f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:18:58.608510 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7953d36e-5385-4ad0-ac53-c68129d2758f admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:18:58.631876 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1109/2246] 10.222.0.23 () {58 vars in 1152 bytes} [Sat Jun 18 05:18:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 05:18:58.638647 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf36dbf-1f82-4f83-ab15-681047c38bc5 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:18:58.653666 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf36dbf-1f82-4f83-ab15-681047c38bc5 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:18:58.661115 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae673bc8-636c-42cc-b1ff-2c64d357cb43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:18:58.661995 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae673bc8-636c-42cc-b1ff-2c64d357cb43 admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:18:58.662524 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae673bc8-636c-42cc-b1ff-2c64d357cb43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:18:58.663350 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae673bc8-636c-42cc-b1ff-2c64d357cb43 admin admin] PATH_INFO: `/v3/projects` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:18:58.664377 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae673bc8-636c-42cc-b1ff-2c64d357cb43 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:18:58.667299 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae673bc8-636c-42cc-b1ff-2c64d357cb43 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:18:58.793448 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1138/2247] 10.222.0.23 () {60 vars in 1152 bytes} [Sat Jun 18 05:18:58 2022] POST /identity/v3/projects => generated 397 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:18:58.798653 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7953d36e-5385-4ad0-ac53-c68129d2758f admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:18:58.815762 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7953d36e-5385-4ad0-ac53-c68129d2758f admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:18:58.899224 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9b4a4d-20bf-4757-a4b1-d10a1c7520e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:18:58.900055 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9b4a4d-20bf-4757-a4b1-d10a1c7520e0 admin admin] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:18:58.900446 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9b4a4d-20bf-4757-a4b1-d10a1c7520e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:18:58.901080 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9b4a4d-20bf-4757-a4b1-d10a1c7520e0 admin admin] PATH_INFO: `/v3/users` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:18:58.902122 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9b4a4d-20bf-4757-a4b1-d10a1c7520e0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:18:58.904485 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9b4a4d-20bf-4757-a4b1-d10a1c7520e0 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:18:59.050027 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1110/2248] 10.222.0.23 () {60 vars in 1143 bytes} [Sat Jun 18 05:18:58 2022] POST /identity/v3/users => generated 299 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:18:59.053961 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae673bc8-636c-42cc-b1ff-2c64d357cb43 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:18:59.066460 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae673bc8-636c-42cc-b1ff-2c64d357cb43 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:18:59.079120 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523203a6-8337-4913-91a3-4e068afc5708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:18:59.080153 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523203a6-8337-4913-91a3-4e068afc5708 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:18:59.080436 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523203a6-8337-4913-91a3-4e068afc5708 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:18:59.080713 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523203a6-8337-4913-91a3-4e068afc5708 admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:18:59.082049 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523203a6-8337-4913-91a3-4e068afc5708 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:18:59.083859 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523203a6-8337-4913-91a3-4e068afc5708 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:18:59.100625 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1139/2249] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:18:59 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:18:59.102311 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9b4a4d-20bf-4757-a4b1-d10a1c7520e0 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:18:59.114350 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9b4a4d-20bf-4757-a4b1-d10a1c7520e0 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:18:59.120376 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d419967-ae5d-4aa5-abdc-355ad61bd981 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:18:59.121447 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d419967-ae5d-4aa5-abdc-355ad61bd981 admin admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:18:59.121889 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d419967-ae5d-4aa5-abdc-355ad61bd981 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:18:59.122264 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d419967-ae5d-4aa5-abdc-355ad61bd981 admin admin] PATH_INFO: `/v3/roles` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:18:59.123189 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d419967-ae5d-4aa5-abdc-355ad61bd981 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:18:59.125007 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d419967-ae5d-4aa5-abdc-355ad61bd981 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:18:59.141429 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1111/2250] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:18:59 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:18:59.143852 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523203a6-8337-4913-91a3-4e068afc5708 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:18:59.157367 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523203a6-8337-4913-91a3-4e068afc5708 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:18:59.162248 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e782bb9e-40fc-453c-ab19-86eb97285bc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:18:59.163126 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e782bb9e-40fc-453c-ab19-86eb97285bc2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:18:59.163405 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e782bb9e-40fc-453c-ab19-86eb97285bc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:18:59.163676 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e782bb9e-40fc-453c-ab19-86eb97285bc2 admin admin] PATH_INFO: `/v3/projects/1eb1066e3eac45b5ab063d4241e374a1/users/9669a3dbc8fb4dc585be46f7bc5584e5/roles/cbf5c4c26898413d93e73b46b41046a3` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:18:59.183068 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e782bb9e-40fc-453c-ab19-86eb97285bc2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1eb1066e3eac45b5ab063d4241e374a1, user_id=9669a3dbc8fb4dc585be46f7bc5584e5, role_id=cbf5c4c26898413d93e73b46b41046a3)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:18:59.186029 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e782bb9e-40fc-453c-ab19-86eb97285bc2 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:18:59.324742 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1140/2251] 10.222.0.23 () {60 vars in 1482 bytes} [Sat Jun 18 05:18:59 2022] PUT /identity/v3/projects/1eb1066e3eac45b5ab063d4241e374a1/users/9669a3dbc8fb4dc585be46f7bc5584e5/roles/cbf5c4c26898413d93e73b46b41046a3 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 18 05:18:59.337009 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1a230d-f655-48b4-94a9-8ab8bac78299 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:18:59.337293 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1a230d-f655-48b4-94a9-8ab8bac78299 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:18:59.337666 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1a230d-f655-48b4-94a9-8ab8bac78299 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:18:59.377775 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-7f1a230d-f655-48b4-94a9-8ab8bac78299 None None] MFA Rules not processed for user `9669a3dbc8fb4dc585be46f7bc5584e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:18:59.377775 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-7f1a230d-f655-48b4-94a9-8ab8bac78299 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:18:59.382466 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:18:59.464049 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1112/2252] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:18:59 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:18:59.474467 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e782bb9e-40fc-453c-ab19-86eb97285bc2 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:18:59.476225 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-e782bb9e-40fc-453c-ab19-86eb97285bc2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:18:59.575466 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e782bb9e-40fc-453c-ab19-86eb97285bc2 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:18:59.579933 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3432ffb6-9335-4ae9-90a6-2b912e75d8d0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9de0428abc8400aa338aba4f7c34c94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:18:59.580591 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3432ffb6-9335-4ae9-90a6-2b912e75d8d0 service cinder] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:18:59.581043 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3432ffb6-9335-4ae9-90a6-2b912e75d8d0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:18:59.581526 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3432ffb6-9335-4ae9-90a6-2b912e75d8d0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:18:59.593743 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3432ffb6-9335-4ae9-90a6-2b912e75d8d0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:18:59.595549 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3432ffb6-9335-4ae9-90a6-2b912e75d8d0 service cinder] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:18:59.607934 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1141/2253] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:18:59 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 18 05:19:09.540575 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=25 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:19:09.602029 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=83,keystone:DELETE=4,keystone:INSERT=3 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:19:11.085371 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1a230d-f655-48b4-94a9-8ab8bac78299 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:19:11.160773 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1a230d-f655-48b4-94a9-8ab8bac78299 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:19:11.165376 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e04c2a1-066b-4758-956f-f14ddc555d96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:19:11.166284 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e04c2a1-066b-4758-956f-f14ddc555d96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:19:11.166573 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e04c2a1-066b-4758-956f-f14ddc555d96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:19:11.166846 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e04c2a1-066b-4758-956f-f14ddc555d96 admin admin] PATH_INFO: `/v3/users/9669a3dbc8fb4dc585be46f7bc5584e5` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:19:11.168038 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e04c2a1-066b-4758-956f-f14ddc555d96 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9669a3dbc8fb4dc585be46f7bc5584e5)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:19:11.170012 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e04c2a1-066b-4758-956f-f14ddc555d96 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:19:11.367190 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3432ffb6-9335-4ae9-90a6-2b912e75d8d0 service cinder] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:19:11.368744 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-3432ffb6-9335-4ae9-90a6-2b912e75d8d0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:19:11.391313 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3432ffb6-9335-4ae9-90a6-2b912e75d8d0 service cinder] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:19:11.397141 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f046230-b21c-430f-8929-a91f71bf008d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:19:11.399036 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f046230-b21c-430f-8929-a91f71bf008d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:19:11.399429 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f046230-b21c-430f-8929-a91f71bf008d admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:19:11.400085 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f046230-b21c-430f-8929-a91f71bf008d admin admin] PATH_INFO: `/v3/users/db60e7b3e2d94e39a9e6de7dbbb1bdea` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:19:11.407582 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f046230-b21c-430f-8929-a91f71bf008d admin admin] RBAC: Authorizing `identity:delete_user(user_id=db60e7b3e2d94e39a9e6de7dbbb1bdea)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:19:11.410602 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f046230-b21c-430f-8929-a91f71bf008d admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:19:11.418173 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-2e04c2a1-066b-4758-956f-f14ddc555d96 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9669a3dbc8fb4dc585be46f7bc5584e5'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:19:11.418996 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-2e04c2a1-066b-4758-956f-f14ddc555d96 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9669a3dbc8fb4dc585be46f7bc5584e5'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:19:11.500711 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-2e04c2a1-066b-4758-956f-f14ddc555d96 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9669a3dbc8fb4dc585be46f7bc5584e5'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:19:11.513538 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-2e04c2a1-066b-4758-956f-f14ddc555d96 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9669a3dbc8fb4dc585be46f7bc5584e5'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:19:11.521198 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-2e04c2a1-066b-4758-956f-f14ddc555d96 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9669a3dbc8fb4dc585be46f7bc5584e5'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:19:11.527569 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1113/2254] 10.222.0.23 () {58 vars in 1223 bytes} [Sat Jun 18 05:19:11 2022] DELETE /identity/v3/users/9669a3dbc8fb4dc585be46f7bc5584e5 => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:19:11.532437 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbb94d1-6d59-44f4-91f7-b7891fcc7ae8 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:19:11.532725 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbb94d1-6d59-44f4-91f7-b7891fcc7ae8 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:19:11.532999 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbb94d1-6d59-44f4-91f7-b7891fcc7ae8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:19:11.550833 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-5cbb94d1-6d59-44f4-91f7-b7891fcc7ae8 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:19:11.552064 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-5cbb94d1-6d59-44f4-91f7-b7891fcc7ae8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:19:11.610141 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-7f046230-b21c-430f-8929-a91f71bf008d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db60e7b3e2d94e39a9e6de7dbbb1bdea'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:19:11.610735 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1114/2255] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:19:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 18 05:19:11.621657 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbb94d1-6d59-44f4-91f7-b7891fcc7ae8 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:19:11.622919 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-7f046230-b21c-430f-8929-a91f71bf008d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db60e7b3e2d94e39a9e6de7dbbb1bdea'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:19:11.623342 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-5cbb94d1-6d59-44f4-91f7-b7891fcc7ae8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:19:11.623622 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-7f046230-b21c-430f-8929-a91f71bf008d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db60e7b3e2d94e39a9e6de7dbbb1bdea'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:19:11.648785 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:19:11.682696 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-7f046230-b21c-430f-8929-a91f71bf008d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db60e7b3e2d94e39a9e6de7dbbb1bdea'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:19:11.692505 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-7f046230-b21c-430f-8929-a91f71bf008d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db60e7b3e2d94e39a9e6de7dbbb1bdea'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:19:11.699497 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1142/2256] 10.222.0.23 () {58 vars in 1223 bytes} [Sat Jun 18 05:19:11 2022] DELETE /identity/v3/users/db60e7b3e2d94e39a9e6de7dbbb1bdea => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 18 05:19:11.704835 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06ffbfd-c2d3-48dc-82a6-ac59f8c7b511 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:19:11.704835 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06ffbfd-c2d3-48dc-82a6-ac59f8c7b511 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:19:11.704835 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06ffbfd-c2d3-48dc-82a6-ac59f8c7b511 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:19:11.706632 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbb94d1-6d59-44f4-91f7-b7891fcc7ae8 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:19:11.727445 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-e06ffbfd-c2d3-48dc-82a6-ac59f8c7b511 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:19:11.728813 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-e06ffbfd-c2d3-48dc-82a6-ac59f8c7b511 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:19:11.769647 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7ed30b-3c72-4d31-88a0-8a26e76c5fcc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c4d7cde8f4ad9aa479bb09e325db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:19:11.770579 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7ed30b-3c72-4d31-88a0-8a26e76c5fcc service neutron] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:19:11.770838 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7ed30b-3c72-4d31-88a0-8a26e76c5fcc service neutron] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:19:11.771225 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7ed30b-3c72-4d31-88a0-8a26e76c5fcc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:19:11.773759 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-9a7ed30b-3c72-4d31-88a0-8a26e76c5fcc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:19:11.799823 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1143/2257] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:19:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 18 05:19:11.808878 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06ffbfd-c2d3-48dc-82a6-ac59f8c7b511 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:19:11.810236 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-e06ffbfd-c2d3-48dc-82a6-ac59f8c7b511 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:19:11.823788 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06ffbfd-c2d3-48dc-82a6-ac59f8c7b511 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:19:11.828712 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb33fffa-e6f5-401a-8ea2-883a352c687f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c4d7cde8f4ad9aa479bb09e325db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:19:11.829362 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb33fffa-e6f5-401a-8ea2-883a352c687f service neutron] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:19:11.829692 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb33fffa-e6f5-401a-8ea2-883a352c687f service neutron] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:19:11.830326 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb33fffa-e6f5-401a-8ea2-883a352c687f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:19:11.841484 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb33fffa-e6f5-401a-8ea2-883a352c687f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:19:11.843062 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb33fffa-e6f5-401a-8ea2-883a352c687f service neutron] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:19:11.852714 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7ed30b-3c72-4d31-88a0-8a26e76c5fcc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:19:11.853316 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1144/2258] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:19:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 18 05:19:11.854187 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7ed30b-3c72-4d31-88a0-8a26e76c5fcc service neutron] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:19:11.865041 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1115/2259] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:19:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:19:12.475375 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb33fffa-e6f5-401a-8ea2-883a352c687f service neutron] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:19:12.477953 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-bb33fffa-e6f5-401a-8ea2-883a352c687f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:19:12.506982 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb33fffa-e6f5-401a-8ea2-883a352c687f service neutron] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:19:12.513733 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2415707a-ca28-4ca8-9585-d6715f9bd036 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:19:12.524423 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2415707a-ca28-4ca8-9585-d6715f9bd036 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:19:12.524908 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2415707a-ca28-4ca8-9585-d6715f9bd036 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:19:12.525200 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2415707a-ca28-4ca8-9585-d6715f9bd036 admin admin] PATH_INFO: `/v3/projects/dceb9a79bfa74dfc923dfd2a38ee3f6f` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:19:12.528239 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2415707a-ca28-4ca8-9585-d6715f9bd036 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dceb9a79bfa74dfc923dfd2a38ee3f6f)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:19:12.531885 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2415707a-ca28-4ca8-9585-d6715f9bd036 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:19:12.537139 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7ed30b-3c72-4d31-88a0-8a26e76c5fcc service neutron] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:19:12.540139 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-9a7ed30b-3c72-4d31-88a0-8a26e76c5fcc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:19:12.578147 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7ed30b-3c72-4d31-88a0-8a26e76c5fcc service neutron] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:19:12.586960 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288e6eca-d78f-4f87-b340-1feb494281ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:19:12.588109 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288e6eca-d78f-4f87-b340-1feb494281ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:19:12.589080 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288e6eca-d78f-4f87-b340-1feb494281ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:19:12.589080 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288e6eca-d78f-4f87-b340-1feb494281ff admin admin] PATH_INFO: `/v3/projects/1eb1066e3eac45b5ab063d4241e374a1` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:19:12.590515 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288e6eca-d78f-4f87-b340-1feb494281ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=1eb1066e3eac45b5ab063d4241e374a1)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:19:12.593204 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288e6eca-d78f-4f87-b340-1feb494281ff admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:19:12.650465 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-2415707a-ca28-4ca8-9585-d6715f9bd036 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dceb9a79bfa74dfc923dfd2a38ee3f6f', 'request_id': 'req-2415707a-ca28-4ca8-9585-d6715f9bd036'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:19:12.654956 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:19:12.658022 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-2415707a-ca28-4ca8-9585-d6715f9bd036 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dceb9a79bfa74dfc923dfd2a38ee3f6f', 'request_id': 'req-2415707a-ca28-4ca8-9585-d6715f9bd036'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:19:12.664506 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-2415707a-ca28-4ca8-9585-d6715f9bd036 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dceb9a79bfa74dfc923dfd2a38ee3f6f', 'request_id': 'req-2415707a-ca28-4ca8-9585-d6715f9bd036'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:19:12.695831 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-288e6eca-d78f-4f87-b340-1feb494281ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1eb1066e3eac45b5ab063d4241e374a1', 'request_id': 'req-288e6eca-d78f-4f87-b340-1feb494281ff'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:19:12.701623 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-288e6eca-d78f-4f87-b340-1feb494281ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1eb1066e3eac45b5ab063d4241e374a1', 'request_id': 'req-288e6eca-d78f-4f87-b340-1feb494281ff'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:19:12.704540 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:19:12.722332 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-2415707a-ca28-4ca8-9585-d6715f9bd036 admin admin] The token cache is being invalidate because project dceb9a79bfa74dfc923dfd2a38ee3f6f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80469) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 18 05:19:12.722625 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-2415707a-ca28-4ca8-9585-d6715f9bd036 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:19:12.723259 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-2415707a-ca28-4ca8-9585-d6715f9bd036 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:19:12.724792 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1145/2260] 10.222.0.23 () {58 vars in 1232 bytes} [Sat Jun 18 05:19:12 2022] DELETE /identity/v3/projects/dceb9a79bfa74dfc923dfd2a38ee3f6f => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:19:12.729562 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35cfdd2-fae2-45c5-a80a-5a54dedde910 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:19:12.729862 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35cfdd2-fae2-45c5-a80a-5a54dedde910 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:19:12.730147 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35cfdd2-fae2-45c5-a80a-5a54dedde910 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:19:12.759837 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-d35cfdd2-fae2-45c5-a80a-5a54dedde910 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:19:12.761104 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-d35cfdd2-fae2-45c5-a80a-5a54dedde910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:19:12.773827 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-288e6eca-d78f-4f87-b340-1feb494281ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1eb1066e3eac45b5ab063d4241e374a1', 'request_id': 'req-288e6eca-d78f-4f87-b340-1feb494281ff'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:19:12.786121 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-288e6eca-d78f-4f87-b340-1feb494281ff admin admin] The token cache is being invalidate because project 1eb1066e3eac45b5ab063d4241e374a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80468) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 18 05:19:12.786411 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-288e6eca-d78f-4f87-b340-1feb494281ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:19:12.791198 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-288e6eca-d78f-4f87-b340-1feb494281ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:19:12.791198 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1116/2261] 10.222.0.23 () {58 vars in 1232 bytes} [Sat Jun 18 05:19:12 2022] DELETE /identity/v3/projects/1eb1066e3eac45b5ab063d4241e374a1 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:19:12.833438 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1146/2262] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:19:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 18 05:19:12.837769 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477cf431-0dea-4847-a82c-cc984e8a9629 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:19:12.843257 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477cf431-0dea-4847-a82c-cc984e8a9629 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:19:12.843257 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477cf431-0dea-4847-a82c-cc984e8a9629 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:19:12.865333 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-477cf431-0dea-4847-a82c-cc984e8a9629 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:19:12.866776 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-477cf431-0dea-4847-a82c-cc984e8a9629 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:19:12.879575 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1117/2263] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:19:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:19:12.886089 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35cfdd2-fae2-45c5-a80a-5a54dedde910 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:19:12.908973 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35cfdd2-fae2-45c5-a80a-5a54dedde910 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:19:12.923100 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f8a96d-9bfb-419f-8509-ca50be41a2fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:19:12.923856 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f8a96d-9bfb-419f-8509-ca50be41a2fc admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:19:12.925278 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f8a96d-9bfb-419f-8509-ca50be41a2fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:19:12.925685 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f8a96d-9bfb-419f-8509-ca50be41a2fc admin admin] PATH_INFO: `/v3/domains` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:19:12.930174 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f8a96d-9bfb-419f-8509-ca50be41a2fc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 05:19:12.930565 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f8a96d-9bfb-419f-8509-ca50be41a2fc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:19:12.932291 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f8a96d-9bfb-419f-8509-ca50be41a2fc admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:19:12.962767 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1147/2264] 10.222.0.23 () {58 vars in 1152 bytes} [Sat Jun 18 05:19:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 05:19:12.963989 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477cf431-0dea-4847-a82c-cc984e8a9629 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:19:12.977087 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477cf431-0dea-4847-a82c-cc984e8a9629 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:19:12.986968 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8fa9e9-d21b-41d7-95cd-f815ef661db3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:19:12.988194 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8fa9e9-d21b-41d7-95cd-f815ef661db3 admin admin] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:19:12.988719 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8fa9e9-d21b-41d7-95cd-f815ef661db3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:19:12.989197 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8fa9e9-d21b-41d7-95cd-f815ef661db3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:19:12.990839 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8fa9e9-d21b-41d7-95cd-f815ef661db3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:19:12.993634 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8fa9e9-d21b-41d7-95cd-f815ef661db3 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:19:13.073867 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1118/2265] 10.222.0.23 () {60 vars in 1152 bytes} [Sat Jun 18 05:19:12 2022] POST /identity/v3/projects => generated 363 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:19:13.075686 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f8a96d-9bfb-419f-8509-ca50be41a2fc admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:19:13.088514 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f8a96d-9bfb-419f-8509-ca50be41a2fc admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:19:13.163405 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5759b75e-fa33-45ad-a725-d072db4b37eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:19:13.164105 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5759b75e-fa33-45ad-a725-d072db4b37eb admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:19:13.164393 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5759b75e-fa33-45ad-a725-d072db4b37eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:19:13.164675 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5759b75e-fa33-45ad-a725-d072db4b37eb admin admin] PATH_INFO: `/v3/users` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:19:13.165369 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5759b75e-fa33-45ad-a725-d072db4b37eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:19:13.167960 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5759b75e-fa33-45ad-a725-d072db4b37eb admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:19:13.282515 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1148/2266] 10.222.0.23 () {60 vars in 1143 bytes} [Sat Jun 18 05:19:13 2022] POST /identity/v3/users => generated 282 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:19:13.286636 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8fa9e9-d21b-41d7-95cd-f815ef661db3 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:19:13.304673 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8fa9e9-d21b-41d7-95cd-f815ef661db3 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:19:13.311274 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c071d20-46e4-43c2-905d-bcf3486b4e09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:19:13.312713 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c071d20-46e4-43c2-905d-bcf3486b4e09 admin admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:19:13.313059 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c071d20-46e4-43c2-905d-bcf3486b4e09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:19:13.320003 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c071d20-46e4-43c2-905d-bcf3486b4e09 admin admin] PATH_INFO: `/v3/roles` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:19:13.320003 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c071d20-46e4-43c2-905d-bcf3486b4e09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:19:13.320003 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c071d20-46e4-43c2-905d-bcf3486b4e09 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:19:13.338952 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1119/2267] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:19:13 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:19:13.342817 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5759b75e-fa33-45ad-a725-d072db4b37eb admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:19:13.353569 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5759b75e-fa33-45ad-a725-d072db4b37eb admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:19:13.359765 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf13347-c09d-4715-8767-c6747901e0a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:19:13.360249 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf13347-c09d-4715-8767-c6747901e0a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:19:13.360603 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf13347-c09d-4715-8767-c6747901e0a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:19:13.360948 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf13347-c09d-4715-8767-c6747901e0a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:19:13.361812 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf13347-c09d-4715-8767-c6747901e0a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:19:13.363589 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf13347-c09d-4715-8767-c6747901e0a4 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:19:13.375597 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1149/2268] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:19:13 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 18 05:19:13.382155 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c071d20-46e4-43c2-905d-bcf3486b4e09 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:19:13.396611 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c071d20-46e4-43c2-905d-bcf3486b4e09 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:19:13.401297 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85345331-2948-48b7-8594-3774a588ce6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:19:13.402238 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85345331-2948-48b7-8594-3774a588ce6f admin admin] REQUEST_METHOD: `PUT` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:19:13.402523 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85345331-2948-48b7-8594-3774a588ce6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:19:13.403054 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85345331-2948-48b7-8594-3774a588ce6f admin admin] PATH_INFO: `/v3/projects/818ede7fcb1a43a4a4a7d3a3fd420297/users/b9c8fa4b5f594f408ea77d23dccb7ba6/roles/cbf5c4c26898413d93e73b46b41046a3` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:19:13.421335 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85345331-2948-48b7-8594-3774a588ce6f admin admin] RBAC: Authorizing `identity:create_grant(project_id=818ede7fcb1a43a4a4a7d3a3fd420297, user_id=b9c8fa4b5f594f408ea77d23dccb7ba6, role_id=cbf5c4c26898413d93e73b46b41046a3)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:19:13.424091 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85345331-2948-48b7-8594-3774a588ce6f admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:19:13.529606 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1120/2269] 10.222.0.23 () {60 vars in 1482 bytes} [Sat Jun 18 05:19:13 2022] PUT /identity/v3/projects/818ede7fcb1a43a4a4a7d3a3fd420297/users/b9c8fa4b5f594f408ea77d23dccb7ba6/roles/cbf5c4c26898413d93e73b46b41046a3 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:19:13.540643 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2f915e-ee45-4cc2-9fab-9f6a7ebba1d6 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:19:13.541136 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2f915e-ee45-4cc2-9fab-9f6a7ebba1d6 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:19:13.541136 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2f915e-ee45-4cc2-9fab-9f6a7ebba1d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:19:13.571391 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-1b2f915e-ee45-4cc2-9fab-9f6a7ebba1d6 None None] MFA Rules not processed for user `b9c8fa4b5f594f408ea77d23dccb7ba6`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:19:13.572772 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-1b2f915e-ee45-4cc2-9fab-9f6a7ebba1d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:19:13.609595 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:19:13.663877 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1150/2270] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:19:13 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:19:13.677929 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85345331-2948-48b7-8594-3774a588ce6f admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:19:13.681620 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-85345331-2948-48b7-8594-3774a588ce6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:19:13.748144 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85345331-2948-48b7-8594-3774a588ce6f admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:19:13.754138 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae30d988-265c-4302-81e2-b47565b30477 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9de0428abc8400aa338aba4f7c34c94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:19:13.756237 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae30d988-265c-4302-81e2-b47565b30477 service cinder] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:19:13.756237 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae30d988-265c-4302-81e2-b47565b30477 service cinder] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:19:13.756237 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae30d988-265c-4302-81e2-b47565b30477 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:19:13.765489 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae30d988-265c-4302-81e2-b47565b30477 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:19:13.767094 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae30d988-265c-4302-81e2-b47565b30477 service cinder] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:19:13.778471 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1121/2271] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:19:13 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:19:15.030169 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c8a336-d62c-4182-892b-cb41d9bcd313 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:19:15.030686 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c8a336-d62c-4182-892b-cb41d9bcd313 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:19:15.030974 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c8a336-d62c-4182-892b-cb41d9bcd313 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:19:15.051260 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-d1c8a336-d62c-4182-892b-cb41d9bcd313 None None] MFA Rules not processed for user `4147695d07c24202b096fb7da07ecf37`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:19:15.052571 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-d1c8a336-d62c-4182-892b-cb41d9bcd313 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:19:15.119263 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1151/2272] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:19:15 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:19:15.811269 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370aabb4-ecd5-4737-b722-517a9453caaa None None] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:19:15.812245 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370aabb4-ecd5-4737-b722-517a9453caaa None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:19:15.817477 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370aabb4-ecd5-4737-b722-517a9453caaa None None] PATH_INFO: `/` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:19:15.817477 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1122/2273] 10.222.0.23 () {54 vars in 921 bytes} [Sat Jun 18 05:19:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 05:19:15.821245 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd29180d-ff29-46cd-9440-1bf155595d4d None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:19:15.821815 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd29180d-ff29-46cd-9440-1bf155595d4d None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:19:15.822346 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd29180d-ff29-46cd-9440-1bf155595d4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:19:15.861252 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-fd29180d-ff29-46cd-9440-1bf155595d4d None None] MFA Rules not processed for user `15e7f45ee0744c1d9602187eac8e8b80`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:19:15.862499 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-fd29180d-ff29-46cd-9440-1bf155595d4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:19:15.942191 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1152/2274] 10.222.0.23 () {58 vars in 1018 bytes} [Sat Jun 18 05:19:15 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 18 05:19:15.954007 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370aabb4-ecd5-4737-b722-517a9453caaa None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:19:15.962486 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-370aabb4-ecd5-4737-b722-517a9453caaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:19:16.085208 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370aabb4-ecd5-4737-b722-517a9453caaa None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:19:16.089816 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5769f0c-968f-4822-a40e-b61b8b1476fc service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4638e5d1fa124667b07054a87c2d353e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:19:16.090660 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5769f0c-968f-4822-a40e-b61b8b1476fc service glance] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:19:16.091198 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5769f0c-968f-4822-a40e-b61b8b1476fc service glance] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:19:16.091486 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5769f0c-968f-4822-a40e-b61b8b1476fc service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:19:16.103701 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5769f0c-968f-4822-a40e-b61b8b1476fc service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:19:16.105524 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5769f0c-968f-4822-a40e-b61b8b1476fc service glance] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:19:16.118738 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1123/2275] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:19:15 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:19:16.319035 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc153dd-8f0b-463f-99ca-7da1e61c79f5 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:19:16.319525 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc153dd-8f0b-463f-99ca-7da1e61c79f5 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:19:16.319809 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc153dd-8f0b-463f-99ca-7da1e61c79f5 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:19:16.321330 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1153/2276] 10.222.0.23 () {54 vars in 921 bytes} [Sat Jun 18 05:19:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 05:19:16.326459 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5b3141-0ca2-4c49-bf4b-9a3d853fac51 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:19:16.326960 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5b3141-0ca2-4c49-bf4b-9a3d853fac51 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:19:16.327158 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5b3141-0ca2-4c49-bf4b-9a3d853fac51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:19:16.346032 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-8a5b3141-0ca2-4c49-bf4b-9a3d853fac51 None None] MFA Rules not processed for user `e78c4d7cde8f4ad9aa479bb09e325db3`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:19:16.347304 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-8a5b3141-0ca2-4c49-bf4b-9a3d853fac51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:19:16.376677 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:DELETE=1,keystone:SELECT=97,keystone:INSERT=2 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:19:16.409547 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1124/2277] 10.222.0.23 () {58 vars in 1018 bytes} [Sat Jun 18 05:19:16 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:19:16.420564 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc153dd-8f0b-463f-99ca-7da1e61c79f5 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:19:16.422312 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-5cc153dd-8f0b-463f-99ca-7da1e61c79f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:19:16.438000 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc153dd-8f0b-463f-99ca-7da1e61c79f5 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:19:16.442281 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4597a735-7fa3-4a91-a5d1-1cb1249b4b98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c4d7cde8f4ad9aa479bb09e325db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:19:16.443215 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4597a735-7fa3-4a91-a5d1-1cb1249b4b98 service neutron] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:19:16.443680 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4597a735-7fa3-4a91-a5d1-1cb1249b4b98 service neutron] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:19:16.444153 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4597a735-7fa3-4a91-a5d1-1cb1249b4b98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:19:16.452681 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4597a735-7fa3-4a91-a5d1-1cb1249b4b98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:19:16.454338 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4597a735-7fa3-4a91-a5d1-1cb1249b4b98 service neutron] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:19:16.463040 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1154/2278] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:19:16 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 18 05:19:16.548614 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2bc7db-617d-4886-8aa3-7c1c52076bcd None None] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:19:16.548911 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2bc7db-617d-4886-8aa3-7c1c52076bcd None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:19:16.549192 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2bc7db-617d-4886-8aa3-7c1c52076bcd None None] PATH_INFO: `/` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:19:16.550662 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1125/2279] 10.222.0.23 () {54 vars in 921 bytes} [Sat Jun 18 05:19:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 05:19:16.556248 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29e215c-6846-45bd-a667-5fc96fc9e5d7 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:19:16.556535 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29e215c-6846-45bd-a667-5fc96fc9e5d7 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:19:16.556810 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29e215c-6846-45bd-a667-5fc96fc9e5d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:19:16.576863 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-d29e215c-6846-45bd-a667-5fc96fc9e5d7 None None] MFA Rules not processed for user `15e7f45ee0744c1d9602187eac8e8b80`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:19:16.578684 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-d29e215c-6846-45bd-a667-5fc96fc9e5d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:19:16.588041 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1155/2280] 10.222.0.23 () {58 vars in 1018 bytes} [Sat Jun 18 05:19:16 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:19:16.599206 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2bc7db-617d-4886-8aa3-7c1c52076bcd None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:19:16.611628 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2bc7db-617d-4886-8aa3-7c1c52076bcd None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:19:16.615430 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f01ea2-0162-43a1-a354-4c77c547e138 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c4d7cde8f4ad9aa479bb09e325db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:19:16.616102 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f01ea2-0162-43a1-a354-4c77c547e138 service neutron] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:19:16.616387 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f01ea2-0162-43a1-a354-4c77c547e138 service neutron] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:19:16.616667 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f01ea2-0162-43a1-a354-4c77c547e138 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:19:16.628561 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f01ea2-0162-43a1-a354-4c77c547e138 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:19:16.630136 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f01ea2-0162-43a1-a354-4c77c547e138 service neutron] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:19:16.642042 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1126/2281] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:19:16 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:19:22.394463 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29e215c-6846-45bd-a667-5fc96fc9e5d7 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:19:22.420531 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29e215c-6846-45bd-a667-5fc96fc9e5d7 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:19:22.429782 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f602db90-54be-41d8-b117-8484d8ddb064 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c4d7cde8f4ad9aa479bb09e325db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:19:22.430637 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f602db90-54be-41d8-b117-8484d8ddb064 service neutron] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:19:22.430916 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f602db90-54be-41d8-b117-8484d8ddb064 service neutron] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:19:22.431191 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f602db90-54be-41d8-b117-8484d8ddb064 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:19:22.433238 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-f602db90-54be-41d8-b117-8484d8ddb064 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:19:22.470442 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:19:22.502794 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f602db90-54be-41d8-b117-8484d8ddb064 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:19:22.504479 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f602db90-54be-41d8-b117-8484d8ddb064 service neutron] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:19:22.518788 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1156/2282] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:19:22 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:19:26.636605 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=15 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:19:29.430758 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee08d3f6-b12f-4973-8e27-8b6db593b02d None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:19:29.431661 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee08d3f6-b12f-4973-8e27-8b6db593b02d None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:19:29.434937 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee08d3f6-b12f-4973-8e27-8b6db593b02d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:19:29.452672 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-ee08d3f6-b12f-4973-8e27-8b6db593b02d None None] MFA Rules not processed for user `e817fafd0c1b4bf9944481cf116f1260`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:19:29.454321 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-ee08d3f6-b12f-4973-8e27-8b6db593b02d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:19:29.538699 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1127/2283] 10.222.0.23 () {58 vars in 1009 bytes} [Sat Jun 18 05:19:29 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:19:29.575777 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f602db90-54be-41d8-b117-8484d8ddb064 service neutron] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:19:29.576419 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-f602db90-54be-41d8-b117-8484d8ddb064 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:19:29.685626 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f602db90-54be-41d8-b117-8484d8ddb064 service neutron] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:19:29.691177 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f073f50-678a-42fe-85c7-fc9ef7c2aaca service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16b49e866398400580ae48aa5a96fc04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:19:29.691898 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f073f50-678a-42fe-85c7-fc9ef7c2aaca service swift] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:19:29.692251 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f073f50-678a-42fe-85c7-fc9ef7c2aaca service swift] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:19:29.692638 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f073f50-678a-42fe-85c7-fc9ef7c2aaca service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:19:29.701425 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f073f50-678a-42fe-85c7-fc9ef7c2aaca service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:19:29.703211 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f073f50-678a-42fe-85c7-fc9ef7c2aaca service swift] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:19:29.711788 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1157/2284] 10.222.0.23 () {60 vars in 1384 bytes} [Sat Jun 18 05:19:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 18 05:19:39.531161 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=28 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:19:39.708387 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=45 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:19:51.582136 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee08d3f6-b12f-4973-8e27-8b6db593b02d None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:19:51.584995 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-ee08d3f6-b12f-4973-8e27-8b6db593b02d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:19:51.713799 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee08d3f6-b12f-4973-8e27-8b6db593b02d None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:19:51.719123 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45666cec-ced8-4f87-b7ec-e1e9d901db7d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5323ffd91f864680b0c0084eb3f44d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:19:51.719769 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45666cec-ced8-4f87-b7ec-e1e9d901db7d service placement] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:19:51.720055 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45666cec-ced8-4f87-b7ec-e1e9d901db7d service placement] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:19:51.720331 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45666cec-ced8-4f87-b7ec-e1e9d901db7d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:19:51.723484 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-45666cec-ced8-4f87-b7ec-e1e9d901db7d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:19:51.737870 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45666cec-ced8-4f87-b7ec-e1e9d901db7d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:19:51.739948 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45666cec-ced8-4f87-b7ec-e1e9d901db7d service placement] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:19:51.751701 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1128/2285] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:19:51 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:19:56.066099 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f073f50-678a-42fe-85c7-fc9ef7c2aaca service swift] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:19:56.067583 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-3f073f50-678a-42fe-85c7-fc9ef7c2aaca service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:19:56.079567 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f073f50-678a-42fe-85c7-fc9ef7c2aaca service swift] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:19:56.083838 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b849b6-c9a1-49b6-a473-d6e017e1f984 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c4d7cde8f4ad9aa479bb09e325db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:19:56.084472 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b849b6-c9a1-49b6-a473-d6e017e1f984 service neutron] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:19:56.084757 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b849b6-c9a1-49b6-a473-d6e017e1f984 service neutron] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:19:56.085036 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b849b6-c9a1-49b6-a473-d6e017e1f984 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:19:56.086876 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-d2b849b6-c9a1-49b6-a473-d6e017e1f984 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:19:56.153611 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b849b6-c9a1-49b6-a473-d6e017e1f984 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:19:56.155261 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b849b6-c9a1-49b6-a473-d6e017e1f984 service neutron] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:19:56.166686 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1158/2286] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:19:56 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 18 05:19:59.255014 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3281269f-7e89-48c9-b1de-286c2c641334 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:19:59.255502 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3281269f-7e89-48c9-b1de-286c2c641334 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:19:59.255719 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3281269f-7e89-48c9-b1de-286c2c641334 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:19:59.273458 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b849b6-c9a1-49b6-a473-d6e017e1f984 service neutron] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:19:59.324374 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-3281269f-7e89-48c9-b1de-286c2c641334 None None] MFA Rules not processed for user `e817fafd0c1b4bf9944481cf116f1260`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:19:59.325654 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-3281269f-7e89-48c9-b1de-286c2c641334 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:19:59.348977 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1129/2287] 10.222.0.23 () {58 vars in 967 bytes} [Sat Jun 18 05:19:59 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:19:59.353143 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b849b6-c9a1-49b6-a473-d6e017e1f984 service neutron] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:19:59.358460 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7470e30-7695-41b8-b180-8021e9e692f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:19:59.368325 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3281269f-7e89-48c9-b1de-286c2c641334 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:19:59.371446 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7470e30-7695-41b8-b180-8021e9e692f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:19:59.371737 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7470e30-7695-41b8-b180-8021e9e692f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:19:59.372313 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7470e30-7695-41b8-b180-8021e9e692f5 admin admin] PATH_INFO: `/v3/users/b9c8fa4b5f594f408ea77d23dccb7ba6` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:19:59.392975 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7470e30-7695-41b8-b180-8021e9e692f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9c8fa4b5f594f408ea77d23dccb7ba6)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:19:59.395926 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7470e30-7695-41b8-b180-8021e9e692f5 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:19:59.401771 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3281269f-7e89-48c9-b1de-286c2c641334 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:19:59.411186 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc98797-a383-44fe-bc5d-222a8c3b71c5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16b49e866398400580ae48aa5a96fc04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:19:59.412259 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc98797-a383-44fe-bc5d-222a8c3b71c5 service swift] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:19:59.412906 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc98797-a383-44fe-bc5d-222a8c3b71c5 service swift] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:19:59.412906 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc98797-a383-44fe-bc5d-222a8c3b71c5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:19:59.426267 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc98797-a383-44fe-bc5d-222a8c3b71c5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:19:59.428511 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc98797-a383-44fe-bc5d-222a8c3b71c5 service swift] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:19:59.449244 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1130/2288] 10.222.0.23 () {60 vars in 1384 bytes} [Sat Jun 18 05:19:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 18 05:19:59.643548 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-c7470e30-7695-41b8-b180-8021e9e692f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9c8fa4b5f594f408ea77d23dccb7ba6'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:19:59.676378 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-c7470e30-7695-41b8-b180-8021e9e692f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9c8fa4b5f594f408ea77d23dccb7ba6'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:19:59.680964 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-c7470e30-7695-41b8-b180-8021e9e692f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9c8fa4b5f594f408ea77d23dccb7ba6'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:19:59.823258 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-c7470e30-7695-41b8-b180-8021e9e692f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9c8fa4b5f594f408ea77d23dccb7ba6'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:19:59.831700 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-c7470e30-7695-41b8-b180-8021e9e692f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9c8fa4b5f594f408ea77d23dccb7ba6'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:19:59.838390 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1159/2289] 10.222.0.23 () {58 vars in 1223 bytes} [Sat Jun 18 05:19:59 2022] DELETE /identity/v3/users/b9c8fa4b5f594f408ea77d23dccb7ba6 => generated 0 bytes in 565 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 18 05:19:59.843080 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ac8561-e4f9-45d5-9f21-6cff7226859e None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:19:59.843371 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ac8561-e4f9-45d5-9f21-6cff7226859e None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:19:59.843694 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ac8561-e4f9-45d5-9f21-6cff7226859e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:19:59.866855 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-12ac8561-e4f9-45d5-9f21-6cff7226859e None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:19:59.867977 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-12ac8561-e4f9-45d5-9f21-6cff7226859e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:19:59.933828 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1131/2290] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:19:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 18 05:19:59.944215 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7470e30-7695-41b8-b180-8021e9e692f5 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:19:59.945470 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-c7470e30-7695-41b8-b180-8021e9e692f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:19:59.960766 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:20:00.009123 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7470e30-7695-41b8-b180-8021e9e692f5 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:20:00.013023 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c4f098-18ba-4683-94ec-1864479338cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c4d7cde8f4ad9aa479bb09e325db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:20:00.013727 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c4f098-18ba-4683-94ec-1864479338cf service neutron] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:20:00.014018 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c4f098-18ba-4683-94ec-1864479338cf service neutron] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:20:00.014307 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c4f098-18ba-4683-94ec-1864479338cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:20:00.025546 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c4f098-18ba-4683-94ec-1864479338cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:20:00.027076 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c4f098-18ba-4683-94ec-1864479338cf service neutron] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:20:00.039792 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1160/2291] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:19:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:20:00.550852 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ac8561-e4f9-45d5-9f21-6cff7226859e None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:20:00.554298 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-12ac8561-e4f9-45d5-9f21-6cff7226859e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:20:00.576156 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ac8561-e4f9-45d5-9f21-6cff7226859e None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:20:00.583428 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603cf10d-d253-4f7e-abb8-7feafefb09fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:20:00.584585 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603cf10d-d253-4f7e-abb8-7feafefb09fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:20:00.585028 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603cf10d-d253-4f7e-abb8-7feafefb09fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:20:00.585726 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603cf10d-d253-4f7e-abb8-7feafefb09fd admin admin] PATH_INFO: `/v3/projects/818ede7fcb1a43a4a4a7d3a3fd420297` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:20:00.587443 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603cf10d-d253-4f7e-abb8-7feafefb09fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=818ede7fcb1a43a4a4a7d3a3fd420297)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:20:00.590330 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603cf10d-d253-4f7e-abb8-7feafefb09fd admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:20:00.593196 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:20:00.715054 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-603cf10d-d253-4f7e-abb8-7feafefb09fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '818ede7fcb1a43a4a4a7d3a3fd420297', 'request_id': 'req-603cf10d-d253-4f7e-abb8-7feafefb09fd'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:20:00.722328 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-603cf10d-d253-4f7e-abb8-7feafefb09fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '818ede7fcb1a43a4a4a7d3a3fd420297', 'request_id': 'req-603cf10d-d253-4f7e-abb8-7feafefb09fd'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:20:00.814968 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-603cf10d-d253-4f7e-abb8-7feafefb09fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '818ede7fcb1a43a4a4a7d3a3fd420297', 'request_id': 'req-603cf10d-d253-4f7e-abb8-7feafefb09fd'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:20:00.832857 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-603cf10d-d253-4f7e-abb8-7feafefb09fd admin admin] The token cache is being invalidate because project 818ede7fcb1a43a4a4a7d3a3fd420297 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80468) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 18 05:20:00.834277 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-603cf10d-d253-4f7e-abb8-7feafefb09fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:20:00.835763 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-603cf10d-d253-4f7e-abb8-7feafefb09fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:20:00.837975 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1132/2292] 10.222.0.23 () {58 vars in 1232 bytes} [Sat Jun 18 05:20:00 2022] DELETE /identity/v3/projects/818ede7fcb1a43a4a4a7d3a3fd420297 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:20:00.844095 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b1975b-1508-4490-9fdc-c7c090a2c2d6 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:20:00.845808 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b1975b-1508-4490-9fdc-c7c090a2c2d6 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:20:00.846997 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b1975b-1508-4490-9fdc-c7c090a2c2d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:20:00.874717 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-e7b1975b-1508-4490-9fdc-c7c090a2c2d6 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:20:00.876953 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-e7b1975b-1508-4490-9fdc-c7c090a2c2d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:20:00.940781 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1161/2293] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:20:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:20:00.951846 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b7d182-e1be-4f21-b7ff-d9de1b93f359 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:20:00.952137 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b7d182-e1be-4f21-b7ff-d9de1b93f359 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:20:00.952413 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b7d182-e1be-4f21-b7ff-d9de1b93f359 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:20:00.975218 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-15b7d182-e1be-4f21-b7ff-d9de1b93f359 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:20:00.977304 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-15b7d182-e1be-4f21-b7ff-d9de1b93f359 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:20:00.987326 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1133/2294] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:20:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:20:00.991555 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b1975b-1508-4490-9fdc-c7c090a2c2d6 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:20:01.001740 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b1975b-1508-4490-9fdc-c7c090a2c2d6 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:20:01.007368 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2c5077-a6cd-423d-9c4c-f727c7ee733d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:20:01.008108 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2c5077-a6cd-423d-9c4c-f727c7ee733d admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:20:01.008400 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2c5077-a6cd-423d-9c4c-f727c7ee733d admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:20:01.014769 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2c5077-a6cd-423d-9c4c-f727c7ee733d admin admin] PATH_INFO: `/v3/domains` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:20:01.014769 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2c5077-a6cd-423d-9c4c-f727c7ee733d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 05:20:01.014769 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2c5077-a6cd-423d-9c4c-f727c7ee733d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:20:01.014769 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2c5077-a6cd-423d-9c4c-f727c7ee733d admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:20:01.023933 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1162/2295] 10.222.0.23 () {58 vars in 1152 bytes} [Sat Jun 18 05:20:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 05:20:01.028499 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b7d182-e1be-4f21-b7ff-d9de1b93f359 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:20:01.038841 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b7d182-e1be-4f21-b7ff-d9de1b93f359 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:20:01.043531 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf67dc7b-038c-4f9c-84dc-5d0d69f90fa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:20:01.044178 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf67dc7b-038c-4f9c-84dc-5d0d69f90fa6 admin admin] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:20:01.044463 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf67dc7b-038c-4f9c-84dc-5d0d69f90fa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:20:01.044741 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf67dc7b-038c-4f9c-84dc-5d0d69f90fa6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:20:01.045479 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf67dc7b-038c-4f9c-84dc-5d0d69f90fa6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:20:01.047512 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf67dc7b-038c-4f9c-84dc-5d0d69f90fa6 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:20:01.132371 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1134/2296] 10.222.0.23 () {60 vars in 1152 bytes} [Sat Jun 18 05:20:01 2022] POST /identity/v3/projects => generated 379 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:20:01.150940 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2c5077-a6cd-423d-9c4c-f727c7ee733d admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:20:01.152546 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2c5077-a6cd-423d-9c4c-f727c7ee733d admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:20:01.214312 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d0db19-90fb-439d-b288-70df18daf1a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:20:01.214855 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d0db19-90fb-439d-b288-70df18daf1a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:20:01.215143 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d0db19-90fb-439d-b288-70df18daf1a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:20:01.215482 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d0db19-90fb-439d-b288-70df18daf1a9 admin admin] PATH_INFO: `/v3/users` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:20:01.216180 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d0db19-90fb-439d-b288-70df18daf1a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:20:01.218161 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d0db19-90fb-439d-b288-70df18daf1a9 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:20:01.317120 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1163/2297] 10.222.0.23 () {60 vars in 1143 bytes} [Sat Jun 18 05:20:01 2022] POST /identity/v3/users => generated 290 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:20:01.321089 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf67dc7b-038c-4f9c-84dc-5d0d69f90fa6 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:20:01.332297 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf67dc7b-038c-4f9c-84dc-5d0d69f90fa6 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:20:01.337251 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329f7c52-ad43-4ea2-a519-280dc1b5200b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:20:01.337932 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329f7c52-ad43-4ea2-a519-280dc1b5200b admin admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:20:01.338220 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329f7c52-ad43-4ea2-a519-280dc1b5200b admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:20:01.338500 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329f7c52-ad43-4ea2-a519-280dc1b5200b admin admin] PATH_INFO: `/v3/roles` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:20:01.339296 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329f7c52-ad43-4ea2-a519-280dc1b5200b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:20:01.341085 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329f7c52-ad43-4ea2-a519-280dc1b5200b admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:20:01.352916 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1135/2298] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:20:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:20:01.357421 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d0db19-90fb-439d-b288-70df18daf1a9 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:20:01.367421 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d0db19-90fb-439d-b288-70df18daf1a9 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:20:01.372221 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1240533a-b359-438b-9aad-d47fec420913 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:20:01.372915 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1240533a-b359-438b-9aad-d47fec420913 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:20:01.373673 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1240533a-b359-438b-9aad-d47fec420913 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:20:01.373954 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1240533a-b359-438b-9aad-d47fec420913 admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:20:01.374723 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1240533a-b359-438b-9aad-d47fec420913 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:20:01.376361 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1240533a-b359-438b-9aad-d47fec420913 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:20:01.387711 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1164/2299] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:20:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:20:01.392111 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329f7c52-ad43-4ea2-a519-280dc1b5200b admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:20:01.404632 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329f7c52-ad43-4ea2-a519-280dc1b5200b admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:20:01.416563 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267a0d09-d37f-4ec9-89ed-310c7cd70127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:20:01.417687 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267a0d09-d37f-4ec9-89ed-310c7cd70127 admin admin] REQUEST_METHOD: `PUT` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:20:01.417979 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267a0d09-d37f-4ec9-89ed-310c7cd70127 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:20:01.418283 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267a0d09-d37f-4ec9-89ed-310c7cd70127 admin admin] PATH_INFO: `/v3/projects/e5e4aece32ff462b9b9a42200d584c1b/users/6d24c8fbdc7246e2af87004f2985a9f5/roles/cbf5c4c26898413d93e73b46b41046a3` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:20:01.440674 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267a0d09-d37f-4ec9-89ed-310c7cd70127 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5e4aece32ff462b9b9a42200d584c1b, user_id=6d24c8fbdc7246e2af87004f2985a9f5, role_id=cbf5c4c26898413d93e73b46b41046a3)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:20:01.444285 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267a0d09-d37f-4ec9-89ed-310c7cd70127 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:20:01.540050 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1136/2300] 10.222.0.23 () {60 vars in 1482 bytes} [Sat Jun 18 05:20:01 2022] PUT /identity/v3/projects/e5e4aece32ff462b9b9a42200d584c1b/users/6d24c8fbdc7246e2af87004f2985a9f5/roles/cbf5c4c26898413d93e73b46b41046a3 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 18 05:20:01.541778 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df68d2af-b868-4812-81c0-0ee1ecd739fb None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:20:01.542068 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df68d2af-b868-4812-81c0-0ee1ecd739fb None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:20:01.542346 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df68d2af-b868-4812-81c0-0ee1ecd739fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:20:01.563211 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-df68d2af-b868-4812-81c0-0ee1ecd739fb None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:20:01.564661 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-df68d2af-b868-4812-81c0-0ee1ecd739fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:20:01.585356 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:20:01.631187 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1165/2301] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:20:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:20:01.640878 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267a0d09-d37f-4ec9-89ed-310c7cd70127 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:20:01.642747 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-267a0d09-d37f-4ec9-89ed-310c7cd70127 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:20:01.718669 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267a0d09-d37f-4ec9-89ed-310c7cd70127 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:20:01.722911 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e622f1f-d8b7-4c0d-9246-36601cc1ded4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c4d7cde8f4ad9aa479bb09e325db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:20:01.723618 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e622f1f-d8b7-4c0d-9246-36601cc1ded4 service neutron] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:20:01.723906 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e622f1f-d8b7-4c0d-9246-36601cc1ded4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:20:01.724183 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e622f1f-d8b7-4c0d-9246-36601cc1ded4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:20:01.735214 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e622f1f-d8b7-4c0d-9246-36601cc1ded4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:20:01.735214 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e622f1f-d8b7-4c0d-9246-36601cc1ded4 service neutron] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:20:01.745787 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1137/2302] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:20:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:20:05.280504 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a249a711-d760-4515-9927-1540a7de2b0b None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:20:05.281014 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a249a711-d760-4515-9927-1540a7de2b0b None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:20:05.281273 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a249a711-d760-4515-9927-1540a7de2b0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:20:05.303992 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-a249a711-d760-4515-9927-1540a7de2b0b None None] MFA Rules not processed for user `216d2b492c55410d9563051142048160`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:20:05.305969 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-a249a711-d760-4515-9927-1540a7de2b0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:20:05.373243 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1166/2303] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:20:05 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:20:07.805865 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26b0a21-b6e1-4f8d-b632-fd9342c73c1e None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:20:07.806389 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26b0a21-b6e1-4f8d-b632-fd9342c73c1e None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:20:07.806484 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26b0a21-b6e1-4f8d-b632-fd9342c73c1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:20:07.836665 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-c26b0a21-b6e1-4f8d-b632-fd9342c73c1e None None] MFA Rules not processed for user `6d24c8fbdc7246e2af87004f2985a9f5`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:20:07.838013 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-c26b0a21-b6e1-4f8d-b632-fd9342c73c1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:20:07.859484 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:20:07.912617 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1138/2304] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:20:07 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:20:07.918141 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a249a711-d760-4515-9927-1540a7de2b0b None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:20:07.930736 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a249a711-d760-4515-9927-1540a7de2b0b None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:20:07.935294 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755b291a-d37d-4cfd-a869-96783fc6e8dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:20:07.935946 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755b291a-d37d-4cfd-a869-96783fc6e8dc admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:20:07.936226 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755b291a-d37d-4cfd-a869-96783fc6e8dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:20:07.936524 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755b291a-d37d-4cfd-a869-96783fc6e8dc admin admin] PATH_INFO: `/v3/projects` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:20:07.937208 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755b291a-d37d-4cfd-a869-96783fc6e8dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:20:07.939288 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755b291a-d37d-4cfd-a869-96783fc6e8dc admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:20:08.038376 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1167/2305] 10.222.0.23 () {60 vars in 1152 bytes} [Sat Jun 18 05:20:07 2022] POST /identity/v3/projects => generated 377 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:20:08.039398 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26b0a21-b6e1-4f8d-b632-fd9342c73c1e None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:20:08.050856 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26b0a21-b6e1-4f8d-b632-fd9342c73c1e None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:20:08.110228 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad533dc-3bf9-40e2-961f-83ecd366a2d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:20:08.110903 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad533dc-3bf9-40e2-961f-83ecd366a2d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:20:08.111183 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad533dc-3bf9-40e2-961f-83ecd366a2d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:20:08.112144 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad533dc-3bf9-40e2-961f-83ecd366a2d7 admin admin] PATH_INFO: `/v3/users` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:20:08.112852 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad533dc-3bf9-40e2-961f-83ecd366a2d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:20:08.114838 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad533dc-3bf9-40e2-961f-83ecd366a2d7 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:20:08.198201 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1139/2306] 10.222.0.23 () {60 vars in 1143 bytes} [Sat Jun 18 05:20:08 2022] POST /identity/v3/users => generated 295 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 18 05:20:08.202147 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755b291a-d37d-4cfd-a869-96783fc6e8dc admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:20:08.213566 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755b291a-d37d-4cfd-a869-96783fc6e8dc admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:20:08.218175 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423eae43-cdf7-4dfd-a361-1cac22adf149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:20:08.218817 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423eae43-cdf7-4dfd-a361-1cac22adf149 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:20:08.219099 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423eae43-cdf7-4dfd-a361-1cac22adf149 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:20:08.219375 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423eae43-cdf7-4dfd-a361-1cac22adf149 admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:20:08.220098 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423eae43-cdf7-4dfd-a361-1cac22adf149 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:20:08.222343 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423eae43-cdf7-4dfd-a361-1cac22adf149 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:20:08.232756 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1168/2307] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:20:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:20:08.237122 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad533dc-3bf9-40e2-961f-83ecd366a2d7 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:20:08.247840 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad533dc-3bf9-40e2-961f-83ecd366a2d7 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:20:08.252605 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ae78ab-6a4e-41d0-961e-905f52c7e5a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:20:08.253545 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ae78ab-6a4e-41d0-961e-905f52c7e5a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:20:08.253830 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ae78ab-6a4e-41d0-961e-905f52c7e5a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:20:08.254110 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ae78ab-6a4e-41d0-961e-905f52c7e5a9 admin admin] PATH_INFO: `/v3/projects/9098492a9e6644edb4d02b2b4b6399ad/users/cd400db0aacb44389379f5c424b81237/roles/21c65f2c51bd4dd283d13507d2714467` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:20:08.270066 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ae78ab-6a4e-41d0-961e-905f52c7e5a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9098492a9e6644edb4d02b2b4b6399ad, user_id=cd400db0aacb44389379f5c424b81237, role_id=21c65f2c51bd4dd283d13507d2714467)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:20:08.272775 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ae78ab-6a4e-41d0-961e-905f52c7e5a9 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:20:08.369254 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1140/2308] 10.222.0.23 () {60 vars in 1482 bytes} [Sat Jun 18 05:20:08 2022] PUT /identity/v3/projects/9098492a9e6644edb4d02b2b4b6399ad/users/cd400db0aacb44389379f5c424b81237/roles/21c65f2c51bd4dd283d13507d2714467 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:20:08.373240 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423eae43-cdf7-4dfd-a361-1cac22adf149 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:20:08.389974 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423eae43-cdf7-4dfd-a361-1cac22adf149 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:20:08.480160 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cbda76-9f5b-4194-ac89-cb8bac9f7392 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:20:08.480842 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cbda76-9f5b-4194-ac89-cb8bac9f7392 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:20:08.481123 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cbda76-9f5b-4194-ac89-cb8bac9f7392 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:20:08.481988 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cbda76-9f5b-4194-ac89-cb8bac9f7392 admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:20:08.482407 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cbda76-9f5b-4194-ac89-cb8bac9f7392 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:20:08.484014 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cbda76-9f5b-4194-ac89-cb8bac9f7392 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:20:08.496723 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1169/2309] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:20:08 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:20:08.500629 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ae78ab-6a4e-41d0-961e-905f52c7e5a9 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:20:08.513911 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ae78ab-6a4e-41d0-961e-905f52c7e5a9 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:20:08.519191 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaeaff7-b9c2-4443-a264-030e28919059 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:20:08.520291 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaeaff7-b9c2-4443-a264-030e28919059 admin admin] REQUEST_METHOD: `PUT` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:20:08.520671 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaeaff7-b9c2-4443-a264-030e28919059 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:20:08.521046 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaeaff7-b9c2-4443-a264-030e28919059 admin admin] PATH_INFO: `/v3/projects/9098492a9e6644edb4d02b2b4b6399ad/users/cd400db0aacb44389379f5c424b81237/roles/21c65f2c51bd4dd283d13507d2714467` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:20:08.523297 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feaeaff7-b9c2-4443-a264-030e28919059 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9098492a9e6644edb4d02b2b4b6399ad, user_id=cd400db0aacb44389379f5c424b81237, role_id=21c65f2c51bd4dd283d13507d2714467)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:20:08.526095 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feaeaff7-b9c2-4443-a264-030e28919059 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:20:08.538795 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1141/2310] 10.222.0.23 () {60 vars in 1482 bytes} [Sat Jun 18 05:20:08 2022] PUT /identity/v3/projects/9098492a9e6644edb4d02b2b4b6399ad/users/cd400db0aacb44389379f5c424b81237/roles/21c65f2c51bd4dd283d13507d2714467 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 18 05:20:08.894950 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffbbedd-7a9f-4fbc-b552-78eb241cc161 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:20:08.895362 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffbbedd-7a9f-4fbc-b552-78eb241cc161 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:20:08.895740 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffbbedd-7a9f-4fbc-b552-78eb241cc161 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:20:08.921111 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-dffbbedd-7a9f-4fbc-b552-78eb241cc161 None None] MFA Rules not processed for user `e817fafd0c1b4bf9944481cf116f1260`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:20:08.927256 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-dffbbedd-7a9f-4fbc-b552-78eb241cc161 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:20:08.965463 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:20:09.024405 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1170/2311] 10.222.0.23 () {58 vars in 1009 bytes} [Sat Jun 18 05:20:08 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 18 05:20:09.033815 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaeaff7-b9c2-4443-a264-030e28919059 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:20:09.037631 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-feaeaff7-b9c2-4443-a264-030e28919059 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:20:09.132159 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:20:09.136928 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaeaff7-b9c2-4443-a264-030e28919059 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:20:09.142714 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228f7623-f3f2-4ab2-a6b7-8e42f81f1178 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16b49e866398400580ae48aa5a96fc04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:20:09.144529 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228f7623-f3f2-4ab2-a6b7-8e42f81f1178 service swift] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:20:09.144529 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228f7623-f3f2-4ab2-a6b7-8e42f81f1178 service swift] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:20:09.144529 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228f7623-f3f2-4ab2-a6b7-8e42f81f1178 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:20:09.157159 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228f7623-f3f2-4ab2-a6b7-8e42f81f1178 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:20:09.157159 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228f7623-f3f2-4ab2-a6b7-8e42f81f1178 service swift] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:20:09.168686 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1142/2312] 10.222.0.23 () {60 vars in 1384 bytes} [Sat Jun 18 05:20:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 18 05:20:17.553034 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9208a415-0c31-423e-ae22-c5b81f59920f None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:20:17.554014 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9208a415-0c31-423e-ae22-c5b81f59920f None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:20:17.554294 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9208a415-0c31-423e-ae22-c5b81f59920f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:20:17.587510 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-9208a415-0c31-423e-ae22-c5b81f59920f None None] MFA Rules not processed for user `cd400db0aacb44389379f5c424b81237`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:20:17.589257 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-9208a415-0c31-423e-ae22-c5b81f59920f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:20:17.694199 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1171/2313] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:20:17 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:20:17.709427 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228f7623-f3f2-4ab2-a6b7-8e42f81f1178 service swift] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:20:17.715990 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-228f7623-f3f2-4ab2-a6b7-8e42f81f1178 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:20:17.883800 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228f7623-f3f2-4ab2-a6b7-8e42f81f1178 service swift] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:20:17.894822 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a210c4e9-7a6c-41e7-b93f-eee402708395 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15e7f45ee0744c1d9602187eac8e8b80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:20:17.895392 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a210c4e9-7a6c-41e7-b93f-eee402708395 service nova] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:20:17.896441 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a210c4e9-7a6c-41e7-b93f-eee402708395 service nova] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:20:17.896734 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a210c4e9-7a6c-41e7-b93f-eee402708395 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:20:17.974591 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a210c4e9-7a6c-41e7-b93f-eee402708395 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:20:17.976269 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a210c4e9-7a6c-41e7-b93f-eee402708395 service nova] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:20:17.992837 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1143/2314] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:20:17 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:20:27.682841 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=45 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:20:27.982711 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=69 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:20:36.977098 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1929b68-ca1b-4d69-a67a-43bd567672b3 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:20:36.977680 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1929b68-ca1b-4d69-a67a-43bd567672b3 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:20:36.977680 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1929b68-ca1b-4d69-a67a-43bd567672b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:20:36.996798 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-f1929b68-ca1b-4d69-a67a-43bd567672b3 None None] MFA Rules not processed for user `e817fafd0c1b4bf9944481cf116f1260`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:20:36.998057 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-f1929b68-ca1b-4d69-a67a-43bd567672b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:20:37.005522 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1172/2315] 10.222.0.23 () {58 vars in 967 bytes} [Sat Jun 18 05:20:36 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:20:37.018347 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a210c4e9-7a6c-41e7-b93f-eee402708395 service nova] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:20:37.030780 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a210c4e9-7a6c-41e7-b93f-eee402708395 service nova] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:20:37.036146 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02c986a-bb6d-4df4-947a-dcb84d8ab27d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16b49e866398400580ae48aa5a96fc04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:20:37.039243 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02c986a-bb6d-4df4-947a-dcb84d8ab27d service swift] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:20:37.039243 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02c986a-bb6d-4df4-947a-dcb84d8ab27d service swift] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:20:37.039243 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02c986a-bb6d-4df4-947a-dcb84d8ab27d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:20:37.047147 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02c986a-bb6d-4df4-947a-dcb84d8ab27d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:20:37.048731 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02c986a-bb6d-4df4-947a-dcb84d8ab27d service swift] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:20:37.058346 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1144/2316] 10.222.0.23 () {60 vars in 1384 bytes} [Sat Jun 18 05:20:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 18 05:20:40.110730 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1929b68-ca1b-4d69-a67a-43bd567672b3 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:20:40.115000 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-f1929b68-ca1b-4d69-a67a-43bd567672b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:20:40.179068 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1929b68-ca1b-4d69-a67a-43bd567672b3 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:20:40.183588 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32df585-75ad-4933-81ae-334960f90862 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:20:40.184450 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32df585-75ad-4933-81ae-334960f90862 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:20:40.184727 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32df585-75ad-4933-81ae-334960f90862 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:20:40.189317 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32df585-75ad-4933-81ae-334960f90862 admin admin] PATH_INFO: `/v3/users/4147695d07c24202b096fb7da07ecf37` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:20:40.189317 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32df585-75ad-4933-81ae-334960f90862 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4147695d07c24202b096fb7da07ecf37)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:20:40.189317 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32df585-75ad-4933-81ae-334960f90862 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:20:40.414968 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-e32df585-75ad-4933-81ae-334960f90862 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4147695d07c24202b096fb7da07ecf37'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:20:40.426862 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-e32df585-75ad-4933-81ae-334960f90862 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4147695d07c24202b096fb7da07ecf37'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:20:40.427553 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-e32df585-75ad-4933-81ae-334960f90862 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4147695d07c24202b096fb7da07ecf37'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:20:40.516191 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-e32df585-75ad-4933-81ae-334960f90862 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4147695d07c24202b096fb7da07ecf37'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:20:40.519052 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-e32df585-75ad-4933-81ae-334960f90862 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4147695d07c24202b096fb7da07ecf37'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:20:40.525773 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1173/2317] 10.222.0.23 () {58 vars in 1223 bytes} [Sat Jun 18 05:20:40 2022] DELETE /identity/v3/users/4147695d07c24202b096fb7da07ecf37 => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:20:40.797779 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02c986a-bb6d-4df4-947a-dcb84d8ab27d service swift] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:20:40.809811 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02c986a-bb6d-4df4-947a-dcb84d8ab27d service swift] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:20:40.868983 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938f1d19-78cc-44aa-962e-044eb2bc3c46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:20:40.869817 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938f1d19-78cc-44aa-962e-044eb2bc3c46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:20:40.870104 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938f1d19-78cc-44aa-962e-044eb2bc3c46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:20:40.870491 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938f1d19-78cc-44aa-962e-044eb2bc3c46 admin admin] PATH_INFO: `/v3/projects/a63728833c434410864d0d7431a9fac2` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:20:40.871695 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938f1d19-78cc-44aa-962e-044eb2bc3c46 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a63728833c434410864d0d7431a9fac2)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:20:40.873644 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938f1d19-78cc-44aa-962e-044eb2bc3c46 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:20:40.979755 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-938f1d19-78cc-44aa-962e-044eb2bc3c46 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a63728833c434410864d0d7431a9fac2', 'request_id': 'req-938f1d19-78cc-44aa-962e-044eb2bc3c46'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:20:40.985339 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-938f1d19-78cc-44aa-962e-044eb2bc3c46 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a63728833c434410864d0d7431a9fac2', 'request_id': 'req-938f1d19-78cc-44aa-962e-044eb2bc3c46'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:20:41.064482 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-938f1d19-78cc-44aa-962e-044eb2bc3c46 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a63728833c434410864d0d7431a9fac2', 'request_id': 'req-938f1d19-78cc-44aa-962e-044eb2bc3c46'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:20:41.076063 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-938f1d19-78cc-44aa-962e-044eb2bc3c46 admin admin] The token cache is being invalidate because project a63728833c434410864d0d7431a9fac2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80468) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 18 05:20:41.076353 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-938f1d19-78cc-44aa-962e-044eb2bc3c46 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:20:41.076984 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-938f1d19-78cc-44aa-962e-044eb2bc3c46 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:20:41.078564 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1145/2318] 10.222.0.23 () {58 vars in 1232 bytes} [Sat Jun 18 05:20:40 2022] DELETE /identity/v3/projects/a63728833c434410864d0d7431a9fac2 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:20:41.083389 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21284eb-748b-48cd-bdb6-725fadcb8d64 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:20:41.083678 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21284eb-748b-48cd-bdb6-725fadcb8d64 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:20:41.083968 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21284eb-748b-48cd-bdb6-725fadcb8d64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:20:41.102735 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-b21284eb-748b-48cd-bdb6-725fadcb8d64 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:20:41.103943 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-b21284eb-748b-48cd-bdb6-725fadcb8d64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:20:41.165223 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1174/2319] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:20:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:20:41.172783 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055c6376-8322-480d-8be3-cf2fc35892bc None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:20:41.173074 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055c6376-8322-480d-8be3-cf2fc35892bc None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:20:41.173354 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055c6376-8322-480d-8be3-cf2fc35892bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:20:41.192105 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-055c6376-8322-480d-8be3-cf2fc35892bc None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:20:41.193430 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-055c6376-8322-480d-8be3-cf2fc35892bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:20:41.202289 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1146/2320] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:20:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:20:41.206153 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21284eb-748b-48cd-bdb6-725fadcb8d64 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:20:41.216560 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21284eb-748b-48cd-bdb6-725fadcb8d64 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:20:41.221259 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cbdd0e-5c85-44bc-9d79-be1709f222a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:20:41.221936 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cbdd0e-5c85-44bc-9d79-be1709f222a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:20:41.222217 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cbdd0e-5c85-44bc-9d79-be1709f222a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:20:41.222491 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cbdd0e-5c85-44bc-9d79-be1709f222a7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:20:41.223253 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cbdd0e-5c85-44bc-9d79-be1709f222a7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 05:20:41.223601 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cbdd0e-5c85-44bc-9d79-be1709f222a7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:20:41.226114 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cbdd0e-5c85-44bc-9d79-be1709f222a7 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:20:41.234578 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:20:41.237962 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1175/2321] 10.222.0.23 () {58 vars in 1152 bytes} [Sat Jun 18 05:20:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 05:20:41.242260 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055c6376-8322-480d-8be3-cf2fc35892bc None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:20:41.251825 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055c6376-8322-480d-8be3-cf2fc35892bc None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:20:41.257542 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23339a0d-a13a-4bea-bcf0-314f720e1899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:20:41.258253 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23339a0d-a13a-4bea-bcf0-314f720e1899 admin admin] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:20:41.258537 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23339a0d-a13a-4bea-bcf0-314f720e1899 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:20:41.258809 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23339a0d-a13a-4bea-bcf0-314f720e1899 admin admin] PATH_INFO: `/v3/projects` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:20:41.259494 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23339a0d-a13a-4bea-bcf0-314f720e1899 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:20:41.261417 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23339a0d-a13a-4bea-bcf0-314f720e1899 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:20:41.325420 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1147/2322] 10.222.0.23 () {60 vars in 1152 bytes} [Sat Jun 18 05:20:41 2022] POST /identity/v3/projects => generated 373 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:20:41.327575 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cbdd0e-5c85-44bc-9d79-be1709f222a7 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:20:41.338721 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cbdd0e-5c85-44bc-9d79-be1709f222a7 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:20:41.395056 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fe918b-bc21-43d8-bca2-b379dcc57ae9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:20:41.395722 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fe918b-bc21-43d8-bca2-b379dcc57ae9 admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:20:41.396004 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fe918b-bc21-43d8-bca2-b379dcc57ae9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:20:41.396279 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fe918b-bc21-43d8-bca2-b379dcc57ae9 admin admin] PATH_INFO: `/v3/users` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:20:41.397724 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fe918b-bc21-43d8-bca2-b379dcc57ae9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:20:41.399665 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fe918b-bc21-43d8-bca2-b379dcc57ae9 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:20:41.482285 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1176/2323] 10.222.0.23 () {60 vars in 1143 bytes} [Sat Jun 18 05:20:41 2022] POST /identity/v3/users => generated 287 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 18 05:20:41.486338 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23339a0d-a13a-4bea-bcf0-314f720e1899 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:20:41.496647 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23339a0d-a13a-4bea-bcf0-314f720e1899 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:20:41.501449 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfe04c9-65ea-4c6a-896f-bbce5d807d59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:20:41.502132 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfe04c9-65ea-4c6a-896f-bbce5d807d59 admin admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:20:41.502413 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfe04c9-65ea-4c6a-896f-bbce5d807d59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:20:41.502691 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfe04c9-65ea-4c6a-896f-bbce5d807d59 admin admin] PATH_INFO: `/v3/roles` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:20:41.503447 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfe04c9-65ea-4c6a-896f-bbce5d807d59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:20:41.505028 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfe04c9-65ea-4c6a-896f-bbce5d807d59 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:20:41.515779 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1148/2324] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:20:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:20:41.519585 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fe918b-bc21-43d8-bca2-b379dcc57ae9 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:20:41.531912 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fe918b-bc21-43d8-bca2-b379dcc57ae9 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:20:41.537054 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2deb64af-594c-4c09-ad29-a26f2e12bb66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:20:41.537984 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2deb64af-594c-4c09-ad29-a26f2e12bb66 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:20:41.538269 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2deb64af-594c-4c09-ad29-a26f2e12bb66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:20:41.538548 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2deb64af-594c-4c09-ad29-a26f2e12bb66 admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:20:41.539278 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2deb64af-594c-4c09-ad29-a26f2e12bb66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:20:41.540930 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2deb64af-594c-4c09-ad29-a26f2e12bb66 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:20:41.555315 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1177/2325] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:20:41 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:20:41.556306 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfe04c9-65ea-4c6a-896f-bbce5d807d59 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:20:41.566227 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfe04c9-65ea-4c6a-896f-bbce5d807d59 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:20:41.571992 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bcf7de-847c-4f82-baf2-fa7dc0314786 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:20:41.572941 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bcf7de-847c-4f82-baf2-fa7dc0314786 admin admin] REQUEST_METHOD: `PUT` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:20:41.573290 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bcf7de-847c-4f82-baf2-fa7dc0314786 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:20:41.573668 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bcf7de-847c-4f82-baf2-fa7dc0314786 admin admin] PATH_INFO: `/v3/projects/e1242924d8394c81a19aa21cac3a8608/users/40903900a6fc4959b315efe42ddd098d/roles/cbf5c4c26898413d93e73b46b41046a3` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:20:41.590115 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bcf7de-847c-4f82-baf2-fa7dc0314786 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1242924d8394c81a19aa21cac3a8608, user_id=40903900a6fc4959b315efe42ddd098d, role_id=cbf5c4c26898413d93e73b46b41046a3)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:20:41.592939 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bcf7de-847c-4f82-baf2-fa7dc0314786 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:20:41.767997 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1149/2326] 10.222.0.23 () {60 vars in 1482 bytes} [Sat Jun 18 05:20:41 2022] PUT /identity/v3/projects/e1242924d8394c81a19aa21cac3a8608/users/40903900a6fc4959b315efe42ddd098d/roles/cbf5c4c26898413d93e73b46b41046a3 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 18 05:20:41.777961 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5488e729-0ec0-4849-b619-94dd549da82d None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:20:41.778251 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5488e729-0ec0-4849-b619-94dd549da82d None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:20:41.778528 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5488e729-0ec0-4849-b619-94dd549da82d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:20:41.806642 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-5488e729-0ec0-4849-b619-94dd549da82d None None] MFA Rules not processed for user `40903900a6fc4959b315efe42ddd098d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:20:41.807848 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-5488e729-0ec0-4849-b619-94dd549da82d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:20:41.871222 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1178/2327] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:20:41 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:20:41.882758 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bcf7de-847c-4f82-baf2-fa7dc0314786 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:20:41.884434 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-75bcf7de-847c-4f82-baf2-fa7dc0314786 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:20:41.902046 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:20:41.964034 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bcf7de-847c-4f82-baf2-fa7dc0314786 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:20:41.967746 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c390a25e-8e5b-469b-8f66-f47df19786b7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9de0428abc8400aa338aba4f7c34c94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:20:41.968380 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c390a25e-8e5b-469b-8f66-f47df19786b7 service cinder] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:20:41.968660 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c390a25e-8e5b-469b-8f66-f47df19786b7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:20:41.968934 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c390a25e-8e5b-469b-8f66-f47df19786b7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:20:41.978236 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c390a25e-8e5b-469b-8f66-f47df19786b7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:20:41.979654 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c390a25e-8e5b-469b-8f66-f47df19786b7 service cinder] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:20:41.988781 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1150/2328] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:20:41 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:20:43.681562 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa725e8b-ec26-4393-be79-21f7e6bd68c0 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:20:43.682064 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa725e8b-ec26-4393-be79-21f7e6bd68c0 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:20:43.682171 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa725e8b-ec26-4393-be79-21f7e6bd68c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:20:43.699452 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-fa725e8b-ec26-4393-be79-21f7e6bd68c0 None None] MFA Rules not processed for user `40903900a6fc4959b315efe42ddd098d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:20:43.700639 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-fa725e8b-ec26-4393-be79-21f7e6bd68c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:20:43.707512 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1179/2329] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:20:43 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:20:45.697682 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960a6660-795c-4f05-8e81-5e7674e6ff0d None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:20:45.698204 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960a6660-795c-4f05-8e81-5e7674e6ff0d None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:20:45.698363 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960a6660-795c-4f05-8e81-5e7674e6ff0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:20:45.715776 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-960a6660-795c-4f05-8e81-5e7674e6ff0d None None] MFA Rules not processed for user `6d24c8fbdc7246e2af87004f2985a9f5`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:20:45.716977 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-960a6660-795c-4f05-8e81-5e7674e6ff0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:20:45.773415 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1151/2330] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:20:45 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 18 05:20:53.694027 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=69,keystone:INSERT=3 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:20:55.765815 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=54 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:21:05.199235 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc942e3-2946-4641-b3cd-b31d38ab69e3 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:21:05.199785 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc942e3-2946-4641-b3cd-b31d38ab69e3 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:21:05.200151 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc942e3-2946-4641-b3cd-b31d38ab69e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:21:05.219182 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-cbc942e3-2946-4641-b3cd-b31d38ab69e3 None None] MFA Rules not processed for user `e817fafd0c1b4bf9944481cf116f1260`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:21:05.220686 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-cbc942e3-2946-4641-b3cd-b31d38ab69e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:21:05.278455 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1180/2331] 10.222.0.23 () {58 vars in 1009 bytes} [Sat Jun 18 05:21:05 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:21:05.291213 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960a6660-795c-4f05-8e81-5e7674e6ff0d None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:21:05.292732 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-960a6660-795c-4f05-8e81-5e7674e6ff0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:21:05.363261 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960a6660-795c-4f05-8e81-5e7674e6ff0d None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:21:05.368675 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9541df-ca8c-4b5b-89e2-3c46293573e9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16b49e866398400580ae48aa5a96fc04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:21:05.369319 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9541df-ca8c-4b5b-89e2-3c46293573e9 service swift] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:21:05.369629 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9541df-ca8c-4b5b-89e2-3c46293573e9 service swift] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:21:05.369899 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9541df-ca8c-4b5b-89e2-3c46293573e9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:21:05.385106 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9541df-ca8c-4b5b-89e2-3c46293573e9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:21:05.386769 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9541df-ca8c-4b5b-89e2-3c46293573e9 service swift] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:21:05.396905 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1152/2332] 10.222.0.23 () {60 vars in 1384 bytes} [Sat Jun 18 05:21:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 18 05:21:06.369705 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033233b8-1d66-42cd-90a1-605558e7313c None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:21:06.370185 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033233b8-1d66-42cd-90a1-605558e7313c None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:21:06.370263 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033233b8-1d66-42cd-90a1-605558e7313c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:21:06.390961 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-033233b8-1d66-42cd-90a1-605558e7313c None None] MFA Rules not processed for user `e817fafd0c1b4bf9944481cf116f1260`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:21:06.392244 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-033233b8-1d66-42cd-90a1-605558e7313c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:21:06.400092 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1181/2333] 10.222.0.23 () {58 vars in 967 bytes} [Sat Jun 18 05:21:06 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:21:06.414149 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9541df-ca8c-4b5b-89e2-3c46293573e9 service swift] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:21:06.426017 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9541df-ca8c-4b5b-89e2-3c46293573e9 service swift] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:21:06.432485 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974b4252-dd7d-4fbd-8beb-9fe3ab2656a8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16b49e866398400580ae48aa5a96fc04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:21:06.433070 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974b4252-dd7d-4fbd-8beb-9fe3ab2656a8 service swift] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:21:06.433615 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974b4252-dd7d-4fbd-8beb-9fe3ab2656a8 service swift] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:21:06.434132 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974b4252-dd7d-4fbd-8beb-9fe3ab2656a8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:21:06.444299 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974b4252-dd7d-4fbd-8beb-9fe3ab2656a8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:21:06.446026 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974b4252-dd7d-4fbd-8beb-9fe3ab2656a8 service swift] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:21:06.454715 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1153/2334] 10.222.0.23 () {60 vars in 1384 bytes} [Sat Jun 18 05:21:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 18 05:21:07.128828 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9d9563-ef82-49be-8b18-59f50f481ecf None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:21:07.129120 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9d9563-ef82-49be-8b18-59f50f481ecf None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:21:07.129406 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9d9563-ef82-49be-8b18-59f50f481ecf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:21:07.149131 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-7d9d9563-ef82-49be-8b18-59f50f481ecf None None] MFA Rules not processed for user `e817fafd0c1b4bf9944481cf116f1260`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:21:07.150432 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-7d9d9563-ef82-49be-8b18-59f50f481ecf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:21:07.157980 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1182/2335] 10.222.0.23 () {58 vars in 1009 bytes} [Sat Jun 18 05:21:07 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:21:07.170633 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974b4252-dd7d-4fbd-8beb-9fe3ab2656a8 service swift] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:21:07.181724 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974b4252-dd7d-4fbd-8beb-9fe3ab2656a8 service swift] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:21:07.187438 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bd5d0b-314f-4f4a-bac7-42a2e2b0bd2f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16b49e866398400580ae48aa5a96fc04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:21:07.188072 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bd5d0b-314f-4f4a-bac7-42a2e2b0bd2f service swift] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:21:07.188356 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bd5d0b-314f-4f4a-bac7-42a2e2b0bd2f service swift] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:21:07.188633 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bd5d0b-314f-4f4a-bac7-42a2e2b0bd2f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:21:07.201096 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bd5d0b-314f-4f4a-bac7-42a2e2b0bd2f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:21:07.202840 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bd5d0b-314f-4f4a-bac7-42a2e2b0bd2f service swift] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:21:07.213628 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1154/2336] 10.222.0.23 () {60 vars in 1384 bytes} [Sat Jun 18 05:21:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 18 05:21:07.843725 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b244450-2095-4cee-a975-f3d19570aae5 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:21:07.844200 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b244450-2095-4cee-a975-f3d19570aae5 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:21:07.849542 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b244450-2095-4cee-a975-f3d19570aae5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:21:07.862669 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-2b244450-2095-4cee-a975-f3d19570aae5 None None] MFA Rules not processed for user `e817fafd0c1b4bf9944481cf116f1260`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:21:07.863878 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-2b244450-2095-4cee-a975-f3d19570aae5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:21:07.871464 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1183/2337] 10.222.0.23 () {58 vars in 967 bytes} [Sat Jun 18 05:21:07 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:21:07.883679 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bd5d0b-314f-4f4a-bac7-42a2e2b0bd2f service swift] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:21:07.894386 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bd5d0b-314f-4f4a-bac7-42a2e2b0bd2f service swift] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:21:07.900129 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f1aa34-0570-4f96-8723-8d455c7026b3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16b49e866398400580ae48aa5a96fc04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:21:07.900829 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f1aa34-0570-4f96-8723-8d455c7026b3 service swift] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:21:07.901109 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f1aa34-0570-4f96-8723-8d455c7026b3 service swift] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:21:07.901406 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f1aa34-0570-4f96-8723-8d455c7026b3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:21:07.909892 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f1aa34-0570-4f96-8723-8d455c7026b3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:21:07.912087 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f1aa34-0570-4f96-8723-8d455c7026b3 service swift] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:21:07.920297 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1155/2338] 10.222.0.23 () {60 vars in 1384 bytes} [Sat Jun 18 05:21:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 18 05:21:08.135458 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58a50a0-176f-4fc0-b2de-f9da74f528e5 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:21:08.135778 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58a50a0-176f-4fc0-b2de-f9da74f528e5 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:21:08.136057 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58a50a0-176f-4fc0-b2de-f9da74f528e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:21:08.154996 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-b58a50a0-176f-4fc0-b2de-f9da74f528e5 None None] MFA Rules not processed for user `e0f3999a062e45748878a3204e3ea456`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:21:08.156204 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-b58a50a0-176f-4fc0-b2de-f9da74f528e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:21:08.227999 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1184/2339] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:21:08 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 18 05:21:09.612187 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f1aa34-0570-4f96-8723-8d455c7026b3 service swift] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:21:09.613653 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-23f1aa34-0570-4f96-8723-8d455c7026b3 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:21:09.672766 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f1aa34-0570-4f96-8723-8d455c7026b3 service swift] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:21:09.676512 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53afea06-5ed2-4e05-aeed-fbe4a46e5a96 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c4d7cde8f4ad9aa479bb09e325db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:21:09.677612 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53afea06-5ed2-4e05-aeed-fbe4a46e5a96 service neutron] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:21:09.677908 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53afea06-5ed2-4e05-aeed-fbe4a46e5a96 service neutron] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:21:09.678198 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53afea06-5ed2-4e05-aeed-fbe4a46e5a96 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:21:09.680000 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-53afea06-5ed2-4e05-aeed-fbe4a46e5a96 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:21:09.744372 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53afea06-5ed2-4e05-aeed-fbe4a46e5a96 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:21:09.745885 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53afea06-5ed2-4e05-aeed-fbe4a46e5a96 service neutron] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:21:09.750881 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:21:09.756221 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1156/2340] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:21:09 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:21:10.645342 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58a50a0-176f-4fc0-b2de-f9da74f528e5 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:21:10.714602 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58a50a0-176f-4fc0-b2de-f9da74f528e5 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:21:10.719395 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8437caf3-aca6-4d86-875f-136e43820bd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:21:10.720264 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8437caf3-aca6-4d86-875f-136e43820bd3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:21:10.720540 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8437caf3-aca6-4d86-875f-136e43820bd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:21:10.720811 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8437caf3-aca6-4d86-875f-136e43820bd3 admin admin] PATH_INFO: `/v3/users/40903900a6fc4959b315efe42ddd098d` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:21:10.722016 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8437caf3-aca6-4d86-875f-136e43820bd3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=40903900a6fc4959b315efe42ddd098d)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:21:10.724557 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8437caf3-aca6-4d86-875f-136e43820bd3 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:21:10.915922 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:21:10.929536 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-8437caf3-aca6-4d86-875f-136e43820bd3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '40903900a6fc4959b315efe42ddd098d'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:21:10.941494 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-8437caf3-aca6-4d86-875f-136e43820bd3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '40903900a6fc4959b315efe42ddd098d'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:21:10.942181 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-8437caf3-aca6-4d86-875f-136e43820bd3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '40903900a6fc4959b315efe42ddd098d'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:21:11.056596 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-8437caf3-aca6-4d86-875f-136e43820bd3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '40903900a6fc4959b315efe42ddd098d'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:21:11.065058 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-8437caf3-aca6-4d86-875f-136e43820bd3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '40903900a6fc4959b315efe42ddd098d'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:21:11.072191 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1185/2341] 10.222.0.23 () {58 vars in 1223 bytes} [Sat Jun 18 05:21:10 2022] DELETE /identity/v3/users/40903900a6fc4959b315efe42ddd098d => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:21:11.076407 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addec638-8077-4a92-89ad-da0c48b65fcf None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:21:11.076916 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addec638-8077-4a92-89ad-da0c48b65fcf None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:21:11.077030 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addec638-8077-4a92-89ad-da0c48b65fcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:21:11.095312 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-addec638-8077-4a92-89ad-da0c48b65fcf None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:21:11.096892 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-addec638-8077-4a92-89ad-da0c48b65fcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:21:11.169282 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1157/2342] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:21:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:21:11.180040 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8437caf3-aca6-4d86-875f-136e43820bd3 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:21:11.182108 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-8437caf3-aca6-4d86-875f-136e43820bd3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:21:11.267532 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8437caf3-aca6-4d86-875f-136e43820bd3 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:21:11.272230 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0a9898-719f-4e1f-b2b9-953f16f4ef6d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c4d7cde8f4ad9aa479bb09e325db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:21:11.272829 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0a9898-719f-4e1f-b2b9-953f16f4ef6d service neutron] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:21:11.273134 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0a9898-719f-4e1f-b2b9-953f16f4ef6d service neutron] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:21:11.273432 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0a9898-719f-4e1f-b2b9-953f16f4ef6d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:21:11.283188 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0a9898-719f-4e1f-b2b9-953f16f4ef6d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:21:11.284675 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0a9898-719f-4e1f-b2b9-953f16f4ef6d service neutron] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:21:11.295213 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1186/2343] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:21:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:21:11.764997 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addec638-8077-4a92-89ad-da0c48b65fcf None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:21:11.766887 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-addec638-8077-4a92-89ad-da0c48b65fcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:21:11.780593 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addec638-8077-4a92-89ad-da0c48b65fcf None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:21:11.785903 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b16f719-0344-42f7-a836-49658694e450 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:21:11.787279 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b16f719-0344-42f7-a836-49658694e450 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:21:11.787843 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b16f719-0344-42f7-a836-49658694e450 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:21:11.788324 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b16f719-0344-42f7-a836-49658694e450 admin admin] PATH_INFO: `/v3/projects/e1242924d8394c81a19aa21cac3a8608` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:21:11.789860 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b16f719-0344-42f7-a836-49658694e450 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e1242924d8394c81a19aa21cac3a8608)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:21:11.792004 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b16f719-0344-42f7-a836-49658694e450 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:21:11.960128 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-3b16f719-0344-42f7-a836-49658694e450 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e1242924d8394c81a19aa21cac3a8608', 'request_id': 'req-3b16f719-0344-42f7-a836-49658694e450'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:21:11.967203 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-3b16f719-0344-42f7-a836-49658694e450 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e1242924d8394c81a19aa21cac3a8608', 'request_id': 'req-3b16f719-0344-42f7-a836-49658694e450'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:21:12.031007 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-3b16f719-0344-42f7-a836-49658694e450 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e1242924d8394c81a19aa21cac3a8608', 'request_id': 'req-3b16f719-0344-42f7-a836-49658694e450'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:21:12.045852 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-3b16f719-0344-42f7-a836-49658694e450 admin admin] The token cache is being invalidate because project e1242924d8394c81a19aa21cac3a8608 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80468) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 18 05:21:12.046061 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-3b16f719-0344-42f7-a836-49658694e450 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:21:12.048015 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-3b16f719-0344-42f7-a836-49658694e450 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:21:12.050801 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1158/2344] 10.222.0.23 () {58 vars in 1232 bytes} [Sat Jun 18 05:21:11 2022] DELETE /identity/v3/projects/e1242924d8394c81a19aa21cac3a8608 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:21:12.059156 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa17635-a9da-4d17-809c-cf2f6de60380 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:21:12.059960 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa17635-a9da-4d17-809c-cf2f6de60380 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:21:12.060464 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa17635-a9da-4d17-809c-cf2f6de60380 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:21:12.093261 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-faa17635-a9da-4d17-809c-cf2f6de60380 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:21:12.094568 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-faa17635-a9da-4d17-809c-cf2f6de60380 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:21:12.169219 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1187/2345] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:21:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:21:12.177299 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc049a60-97a7-4ac5-8666-919c5e04a76d None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:21:12.178524 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc049a60-97a7-4ac5-8666-919c5e04a76d None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:21:12.179010 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc049a60-97a7-4ac5-8666-919c5e04a76d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:21:12.202841 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-bc049a60-97a7-4ac5-8666-919c5e04a76d None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:21:12.204235 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-bc049a60-97a7-4ac5-8666-919c5e04a76d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:21:12.217433 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1159/2346] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:21:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:21:12.219754 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa17635-a9da-4d17-809c-cf2f6de60380 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:21:12.234904 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa17635-a9da-4d17-809c-cf2f6de60380 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:21:12.240774 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a4ce3d-6cfe-4057-b98c-5b070e7fbe59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:21:12.241519 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a4ce3d-6cfe-4057-b98c-5b070e7fbe59 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:21:12.241807 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a4ce3d-6cfe-4057-b98c-5b070e7fbe59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:21:12.242081 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a4ce3d-6cfe-4057-b98c-5b070e7fbe59 admin admin] PATH_INFO: `/v3/domains` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:21:12.242819 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a4ce3d-6cfe-4057-b98c-5b070e7fbe59 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 05:21:12.243130 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a4ce3d-6cfe-4057-b98c-5b070e7fbe59 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:21:12.244810 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a4ce3d-6cfe-4057-b98c-5b070e7fbe59 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:21:12.264534 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1188/2347] 10.222.0.23 () {58 vars in 1152 bytes} [Sat Jun 18 05:21:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 05:21:12.265707 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc049a60-97a7-4ac5-8666-919c5e04a76d None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:21:12.278664 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc049a60-97a7-4ac5-8666-919c5e04a76d None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:21:12.284518 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a8538c-00ac-4a24-82d0-889649e96b13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:21:12.285513 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a8538c-00ac-4a24-82d0-889649e96b13 admin admin] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:21:12.285965 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a8538c-00ac-4a24-82d0-889649e96b13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:21:12.286573 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a8538c-00ac-4a24-82d0-889649e96b13 admin admin] PATH_INFO: `/v3/projects` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:21:12.287446 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a8538c-00ac-4a24-82d0-889649e96b13 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:21:12.290371 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a8538c-00ac-4a24-82d0-889649e96b13 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:21:12.591786 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1160/2348] 10.222.0.23 () {60 vars in 1152 bytes} [Sat Jun 18 05:21:12 2022] POST /identity/v3/projects => generated 391 bytes in 327 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:21:12.595989 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a4ce3d-6cfe-4057-b98c-5b070e7fbe59 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:21:12.607102 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a4ce3d-6cfe-4057-b98c-5b070e7fbe59 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:21:12.658931 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:21:12.683751 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437e64f8-f508-408a-ac5a-f273852039fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:21:12.684572 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437e64f8-f508-408a-ac5a-f273852039fe admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:21:12.684963 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437e64f8-f508-408a-ac5a-f273852039fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:21:12.685356 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437e64f8-f508-408a-ac5a-f273852039fe admin admin] PATH_INFO: `/v3/users` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:21:12.686298 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437e64f8-f508-408a-ac5a-f273852039fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:21:12.688786 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437e64f8-f508-408a-ac5a-f273852039fe admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:21:12.792215 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1189/2349] 10.222.0.23 () {60 vars in 1143 bytes} [Sat Jun 18 05:21:12 2022] POST /identity/v3/users => generated 296 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:21:12.794288 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a8538c-00ac-4a24-82d0-889649e96b13 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:21:12.805163 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a8538c-00ac-4a24-82d0-889649e96b13 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:21:12.810853 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beabb49c-5928-4a0d-af15-c14aec1d90a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:21:12.811635 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beabb49c-5928-4a0d-af15-c14aec1d90a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:21:12.812016 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beabb49c-5928-4a0d-af15-c14aec1d90a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:21:12.812392 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beabb49c-5928-4a0d-af15-c14aec1d90a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:21:12.813244 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beabb49c-5928-4a0d-af15-c14aec1d90a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:21:12.814971 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beabb49c-5928-4a0d-af15-c14aec1d90a6 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:21:12.832383 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1161/2350] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:21:12 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:21:12.834624 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437e64f8-f508-408a-ac5a-f273852039fe admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:21:12.851389 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437e64f8-f508-408a-ac5a-f273852039fe admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:21:12.862100 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b504106-fd66-48d6-bdbd-298da417e003 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:21:12.863119 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b504106-fd66-48d6-bdbd-298da417e003 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:21:12.863409 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b504106-fd66-48d6-bdbd-298da417e003 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:21:12.863688 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b504106-fd66-48d6-bdbd-298da417e003 admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:21:12.864559 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b504106-fd66-48d6-bdbd-298da417e003 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:21:12.866363 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b504106-fd66-48d6-bdbd-298da417e003 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:21:12.880529 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1190/2351] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:21:12 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 18 05:21:12.882695 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beabb49c-5928-4a0d-af15-c14aec1d90a6 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:21:12.895231 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beabb49c-5928-4a0d-af15-c14aec1d90a6 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:21:12.901994 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b5d99b-5652-465e-977b-6ff4960aad97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:21:12.903914 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b5d99b-5652-465e-977b-6ff4960aad97 admin admin] REQUEST_METHOD: `PUT` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:21:12.903914 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b5d99b-5652-465e-977b-6ff4960aad97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:21:12.903914 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b5d99b-5652-465e-977b-6ff4960aad97 admin admin] PATH_INFO: `/v3/projects/6de9d930878c47299a4aebfa61f370fe/users/5e7f9682cafb41c0ba8cbaa7942fbff4/roles/cbf5c4c26898413d93e73b46b41046a3` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:21:12.919951 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b5d99b-5652-465e-977b-6ff4960aad97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6de9d930878c47299a4aebfa61f370fe, user_id=5e7f9682cafb41c0ba8cbaa7942fbff4, role_id=cbf5c4c26898413d93e73b46b41046a3)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:21:12.922664 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b5d99b-5652-465e-977b-6ff4960aad97 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:21:13.051158 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1162/2352] 10.222.0.23 () {60 vars in 1482 bytes} [Sat Jun 18 05:21:12 2022] PUT /identity/v3/projects/6de9d930878c47299a4aebfa61f370fe/users/5e7f9682cafb41c0ba8cbaa7942fbff4/roles/cbf5c4c26898413d93e73b46b41046a3 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:21:13.062193 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb78e499-d558-400e-8eb1-caf57a76277b None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:21:13.063043 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb78e499-d558-400e-8eb1-caf57a76277b None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:21:13.065276 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb78e499-d558-400e-8eb1-caf57a76277b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:21:13.142947 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-bb78e499-d558-400e-8eb1-caf57a76277b None None] MFA Rules not processed for user `5e7f9682cafb41c0ba8cbaa7942fbff4`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:21:13.144447 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-bb78e499-d558-400e-8eb1-caf57a76277b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:21:13.279516 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1191/2353] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:21:13 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:21:13.290680 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b5d99b-5652-465e-977b-6ff4960aad97 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:21:13.295397 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-82b5d99b-5652-465e-977b-6ff4960aad97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:21:13.326729 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:21:13.374587 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b5d99b-5652-465e-977b-6ff4960aad97 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:21:13.382219 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ed88a0-8732-49bd-8ecc-e03654d0f4e7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9de0428abc8400aa338aba4f7c34c94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:21:13.383120 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ed88a0-8732-49bd-8ecc-e03654d0f4e7 service cinder] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:21:13.383574 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ed88a0-8732-49bd-8ecc-e03654d0f4e7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:21:13.384142 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ed88a0-8732-49bd-8ecc-e03654d0f4e7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:21:13.402021 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ed88a0-8732-49bd-8ecc-e03654d0f4e7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:21:13.403734 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ed88a0-8732-49bd-8ecc-e03654d0f4e7 service cinder] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:21:13.418020 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1163/2354] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:21:13 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:21:23.274395 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=48,keystone:INSERT=3 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:21:23.693637 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=17 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:21:29.291529 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb78e499-d558-400e-8eb1-caf57a76277b None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:21:29.293035 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-bb78e499-d558-400e-8eb1-caf57a76277b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:21:29.384719 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb78e499-d558-400e-8eb1-caf57a76277b None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:21:29.398206 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269d77dd-0aea-49f6-ba17-36c2330005d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c4d7cde8f4ad9aa479bb09e325db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:21:29.404961 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269d77dd-0aea-49f6-ba17-36c2330005d8 service neutron] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:21:29.404961 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269d77dd-0aea-49f6-ba17-36c2330005d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:21:29.404961 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269d77dd-0aea-49f6-ba17-36c2330005d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:21:29.404961 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-269d77dd-0aea-49f6-ba17-36c2330005d8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:21:29.512075 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269d77dd-0aea-49f6-ba17-36c2330005d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:21:29.513812 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269d77dd-0aea-49f6-ba17-36c2330005d8 service neutron] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:21:29.534116 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1192/2355] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:21:29 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:21:30.766820 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb819e75-0593-4c16-8469-92ff152108dc None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:21:30.767299 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb819e75-0593-4c16-8469-92ff152108dc None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:21:30.767564 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb819e75-0593-4c16-8469-92ff152108dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:21:30.786723 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-bb819e75-0593-4c16-8469-92ff152108dc None None] MFA Rules not processed for user `e817fafd0c1b4bf9944481cf116f1260`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:21:30.788089 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-bb819e75-0593-4c16-8469-92ff152108dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:21:30.856206 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1164/2356] 10.222.0.23 () {58 vars in 967 bytes} [Sat Jun 18 05:21:30 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:21:30.886987 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269d77dd-0aea-49f6-ba17-36c2330005d8 service neutron] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:21:30.888601 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-269d77dd-0aea-49f6-ba17-36c2330005d8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:21:30.988794 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269d77dd-0aea-49f6-ba17-36c2330005d8 service neutron] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:21:30.995488 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b5c78e-0e14-4123-b8f6-5185238ed9a0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16b49e866398400580ae48aa5a96fc04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:21:30.996178 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b5c78e-0e14-4123-b8f6-5185238ed9a0 service swift] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:21:30.996450 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b5c78e-0e14-4123-b8f6-5185238ed9a0 service swift] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:21:30.996724 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b5c78e-0e14-4123-b8f6-5185238ed9a0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:21:31.007379 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b5c78e-0e14-4123-b8f6-5185238ed9a0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:21:31.009009 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b5c78e-0e14-4123-b8f6-5185238ed9a0 service swift] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:21:31.018517 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1193/2357] 10.222.0.23 () {60 vars in 1384 bytes} [Sat Jun 18 05:21:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 18 05:21:34.529882 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb819e75-0593-4c16-8469-92ff152108dc None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:21:34.612344 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb819e75-0593-4c16-8469-92ff152108dc None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:21:34.617568 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7897f02-ce8b-4124-87d4-24b5f00a847c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:21:34.618547 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7897f02-ce8b-4124-87d4-24b5f00a847c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:21:34.618831 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7897f02-ce8b-4124-87d4-24b5f00a847c admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:21:34.620509 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7897f02-ce8b-4124-87d4-24b5f00a847c admin admin] PATH_INFO: `/v3/users/5e7f9682cafb41c0ba8cbaa7942fbff4` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:21:34.622341 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7897f02-ce8b-4124-87d4-24b5f00a847c admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e7f9682cafb41c0ba8cbaa7942fbff4)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:21:34.624306 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7897f02-ce8b-4124-87d4-24b5f00a847c admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:21:34.793642 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-b7897f02-ce8b-4124-87d4-24b5f00a847c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e7f9682cafb41c0ba8cbaa7942fbff4'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:21:34.794335 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-b7897f02-ce8b-4124-87d4-24b5f00a847c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e7f9682cafb41c0ba8cbaa7942fbff4'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:21:34.875221 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-b7897f02-ce8b-4124-87d4-24b5f00a847c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e7f9682cafb41c0ba8cbaa7942fbff4'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:21:34.889281 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-b7897f02-ce8b-4124-87d4-24b5f00a847c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e7f9682cafb41c0ba8cbaa7942fbff4'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:21:34.896330 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-b7897f02-ce8b-4124-87d4-24b5f00a847c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e7f9682cafb41c0ba8cbaa7942fbff4'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:21:34.903429 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1165/2358] 10.222.0.23 () {58 vars in 1223 bytes} [Sat Jun 18 05:21:34 2022] DELETE /identity/v3/users/5e7f9682cafb41c0ba8cbaa7942fbff4 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:21:34.908294 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9910e929-4d96-43a6-82a9-11e08a6e7879 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:21:34.908761 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9910e929-4d96-43a6-82a9-11e08a6e7879 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:21:34.908970 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9910e929-4d96-43a6-82a9-11e08a6e7879 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:21:34.926881 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-9910e929-4d96-43a6-82a9-11e08a6e7879 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:21:34.928133 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-9910e929-4d96-43a6-82a9-11e08a6e7879 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:21:34.954498 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:21:34.996622 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1194/2359] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:21:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:21:35.006303 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7897f02-ce8b-4124-87d4-24b5f00a847c admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:21:35.008786 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-b7897f02-ce8b-4124-87d4-24b5f00a847c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:21:35.025119 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9910e929-4d96-43a6-82a9-11e08a6e7879 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:21:35.026874 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-9910e929-4d96-43a6-82a9-11e08a6e7879 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:21:35.047496 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:21:35.087549 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7897f02-ce8b-4124-87d4-24b5f00a847c admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:21:35.093410 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b75300b-8e3e-4786-82f7-8e299a41da09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c4d7cde8f4ad9aa479bb09e325db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:21:35.094781 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b75300b-8e3e-4786-82f7-8e299a41da09 service neutron] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:21:35.095073 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b75300b-8e3e-4786-82f7-8e299a41da09 service neutron] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:21:35.095405 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b75300b-8e3e-4786-82f7-8e299a41da09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:21:35.097268 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9910e929-4d96-43a6-82a9-11e08a6e7879 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:21:35.102081 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4175c43-9c0b-4a61-91ae-ccaf759d5358 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c4d7cde8f4ad9aa479bb09e325db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:21:35.102709 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4175c43-9c0b-4a61-91ae-ccaf759d5358 service neutron] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:21:35.103001 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4175c43-9c0b-4a61-91ae-ccaf759d5358 service neutron] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:21:35.103333 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4175c43-9c0b-4a61-91ae-ccaf759d5358 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:21:35.105134 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-c4175c43-9c0b-4a61-91ae-ccaf759d5358 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:21:35.107896 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b75300b-8e3e-4786-82f7-8e299a41da09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:21:35.109549 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b75300b-8e3e-4786-82f7-8e299a41da09 service neutron] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:21:35.125196 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1166/2360] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:21:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:21:35.189628 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4175c43-9c0b-4a61-91ae-ccaf759d5358 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:21:35.191339 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4175c43-9c0b-4a61-91ae-ccaf759d5358 service neutron] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:21:35.205280 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1195/2361] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:21:35 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:21:35.983300 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b75300b-8e3e-4786-82f7-8e299a41da09 service neutron] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:21:35.986880 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-4b75300b-8e3e-4786-82f7-8e299a41da09 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:21:36.010732 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b75300b-8e3e-4786-82f7-8e299a41da09 service neutron] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:21:36.011682 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9869d7a2-4fea-4d32-8fe5-55d5cc623e97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:21:36.012649 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9869d7a2-4fea-4d32-8fe5-55d5cc623e97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:21:36.012941 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9869d7a2-4fea-4d32-8fe5-55d5cc623e97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:21:36.013211 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9869d7a2-4fea-4d32-8fe5-55d5cc623e97 admin admin] PATH_INFO: `/v3/projects/6de9d930878c47299a4aebfa61f370fe` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:21:36.014596 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9869d7a2-4fea-4d32-8fe5-55d5cc623e97 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6de9d930878c47299a4aebfa61f370fe)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:21:36.017286 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9869d7a2-4fea-4d32-8fe5-55d5cc623e97 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:21:36.155539 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-9869d7a2-4fea-4d32-8fe5-55d5cc623e97 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6de9d930878c47299a4aebfa61f370fe', 'request_id': 'req-9869d7a2-4fea-4d32-8fe5-55d5cc623e97'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:21:36.161125 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-9869d7a2-4fea-4d32-8fe5-55d5cc623e97 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6de9d930878c47299a4aebfa61f370fe', 'request_id': 'req-9869d7a2-4fea-4d32-8fe5-55d5cc623e97'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:21:36.252819 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-9869d7a2-4fea-4d32-8fe5-55d5cc623e97 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6de9d930878c47299a4aebfa61f370fe', 'request_id': 'req-9869d7a2-4fea-4d32-8fe5-55d5cc623e97'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:21:36.268795 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-9869d7a2-4fea-4d32-8fe5-55d5cc623e97 admin admin] The token cache is being invalidate because project 6de9d930878c47299a4aebfa61f370fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80468) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 18 05:21:36.269194 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-9869d7a2-4fea-4d32-8fe5-55d5cc623e97 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:21:36.270056 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-9869d7a2-4fea-4d32-8fe5-55d5cc623e97 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:21:36.282551 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1167/2362] 10.222.0.23 () {58 vars in 1232 bytes} [Sat Jun 18 05:21:35 2022] DELETE /identity/v3/projects/6de9d930878c47299a4aebfa61f370fe => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:21:40.596445 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4175c43-9c0b-4a61-91ae-ccaf759d5358 service neutron] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:21:40.598279 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-c4175c43-9c0b-4a61-91ae-ccaf759d5358 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:21:40.672788 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4175c43-9c0b-4a61-91ae-ccaf759d5358 service neutron] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:21:40.688759 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:21:40.740396 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ad782d-c951-458a-a666-acbab6817424 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5323ffd91f864680b0c0084eb3f44d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:21:40.742119 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ad782d-c951-458a-a666-acbab6817424 service placement] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:21:40.742399 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ad782d-c951-458a-a666-acbab6817424 service placement] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:21:40.742680 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ad782d-c951-458a-a666-acbab6817424 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:21:40.744495 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-55ad782d-c951-458a-a666-acbab6817424 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:21:40.756440 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ad782d-c951-458a-a666-acbab6817424 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:21:40.758183 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ad782d-c951-458a-a666-acbab6817424 service placement] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:21:40.768269 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1196/2363] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:21:40 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:21:40.789040 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9869d7a2-4fea-4d32-8fe5-55d5cc623e97 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:21:40.790599 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-9869d7a2-4fea-4d32-8fe5-55d5cc623e97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:21:40.859447 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9869d7a2-4fea-4d32-8fe5-55d5cc623e97 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:21:40.864240 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c60646b-b9a4-4732-94b7-0d65eac48056 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:21:40.865102 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c60646b-b9a4-4732-94b7-0d65eac48056 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:21:40.865442 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c60646b-b9a4-4732-94b7-0d65eac48056 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:21:40.865716 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c60646b-b9a4-4732-94b7-0d65eac48056 admin admin] PATH_INFO: `/v3/users/113fba3872764b02bcbcc69299d8768f` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:21:40.866932 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c60646b-b9a4-4732-94b7-0d65eac48056 admin admin] RBAC: Authorizing `identity:delete_user(user_id=113fba3872764b02bcbcc69299d8768f)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:21:40.868837 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c60646b-b9a4-4732-94b7-0d65eac48056 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:21:41.066930 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-4c60646b-b9a4-4732-94b7-0d65eac48056 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '113fba3872764b02bcbcc69299d8768f'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:21:41.067526 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-4c60646b-b9a4-4732-94b7-0d65eac48056 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '113fba3872764b02bcbcc69299d8768f'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:21:41.071642 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:21:41.132561 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-4c60646b-b9a4-4732-94b7-0d65eac48056 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '113fba3872764b02bcbcc69299d8768f'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:21:41.144434 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-4c60646b-b9a4-4732-94b7-0d65eac48056 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '113fba3872764b02bcbcc69299d8768f'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:21:41.152675 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-4c60646b-b9a4-4732-94b7-0d65eac48056 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '113fba3872764b02bcbcc69299d8768f'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:21:41.158940 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1168/2364] 10.222.0.23 () {58 vars in 1223 bytes} [Sat Jun 18 05:21:40 2022] DELETE /identity/v3/users/113fba3872764b02bcbcc69299d8768f => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:21:41.389801 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ad782d-c951-458a-a666-acbab6817424 service placement] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:21:41.402400 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ad782d-c951-458a-a666-acbab6817424 service placement] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:21:41.462860 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddbdcc7-3b25-4d21-93b4-433460bd745b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:21:41.463720 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddbdcc7-3b25-4d21-93b4-433460bd745b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:21:41.463998 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddbdcc7-3b25-4d21-93b4-433460bd745b admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:21:41.464269 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddbdcc7-3b25-4d21-93b4-433460bd745b admin admin] PATH_INFO: `/v3/projects/3a833f00ba294564a66b9eefbc49147e` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:21:41.465621 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddbdcc7-3b25-4d21-93b4-433460bd745b admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a833f00ba294564a66b9eefbc49147e)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:21:41.467520 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddbdcc7-3b25-4d21-93b4-433460bd745b admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:21:41.585141 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-8ddbdcc7-3b25-4d21-93b4-433460bd745b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a833f00ba294564a66b9eefbc49147e', 'request_id': 'req-8ddbdcc7-3b25-4d21-93b4-433460bd745b'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:21:41.590777 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-8ddbdcc7-3b25-4d21-93b4-433460bd745b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a833f00ba294564a66b9eefbc49147e', 'request_id': 'req-8ddbdcc7-3b25-4d21-93b4-433460bd745b'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:21:41.597107 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-8ddbdcc7-3b25-4d21-93b4-433460bd745b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a833f00ba294564a66b9eefbc49147e', 'request_id': 'req-8ddbdcc7-3b25-4d21-93b4-433460bd745b'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:21:41.669288 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-8ddbdcc7-3b25-4d21-93b4-433460bd745b admin admin] The token cache is being invalidate because project 3a833f00ba294564a66b9eefbc49147e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80469) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 18 05:21:41.669622 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-8ddbdcc7-3b25-4d21-93b4-433460bd745b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:21:41.670249 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-8ddbdcc7-3b25-4d21-93b4-433460bd745b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:21:41.671801 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1197/2365] 10.222.0.23 () {58 vars in 1232 bytes} [Sat Jun 18 05:21:41 2022] DELETE /identity/v3/projects/3a833f00ba294564a66b9eefbc49147e => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:21:41.675803 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c60646b-b9a4-4732-94b7-0d65eac48056 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:21:41.677182 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-4c60646b-b9a4-4732-94b7-0d65eac48056 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:21:41.690401 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c60646b-b9a4-4732-94b7-0d65eac48056 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:21:41.746153 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce1e2c1-d4fb-4200-b392-6a647196b2da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:21:41.747011 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce1e2c1-d4fb-4200-b392-6a647196b2da admin admin] REQUEST_METHOD: `DELETE` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:21:41.747296 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce1e2c1-d4fb-4200-b392-6a647196b2da admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:21:41.747573 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce1e2c1-d4fb-4200-b392-6a647196b2da admin admin] PATH_INFO: `/v3/users/2d63e07595cb4ebcafb24f6928f06f61` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:21:41.748759 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce1e2c1-d4fb-4200-b392-6a647196b2da admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d63e07595cb4ebcafb24f6928f06f61)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:21:41.750723 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce1e2c1-d4fb-4200-b392-6a647196b2da admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:21:41.780244 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddbdcc7-3b25-4d21-93b4-433460bd745b admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:21:41.781683 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-8ddbdcc7-3b25-4d21-93b4-433460bd745b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:21:41.824129 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:21:41.855827 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddbdcc7-3b25-4d21-93b4-433460bd745b admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:21:41.861531 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324ef196-82b2-4462-9192-4eec7f63b542 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15e7f45ee0744c1d9602187eac8e8b80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:21:41.862174 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324ef196-82b2-4462-9192-4eec7f63b542 service nova] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:21:41.862461 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324ef196-82b2-4462-9192-4eec7f63b542 service nova] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:21:41.862737 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324ef196-82b2-4462-9192-4eec7f63b542 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:21:41.864559 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-324ef196-82b2-4462-9192-4eec7f63b542 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:21:41.922546 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324ef196-82b2-4462-9192-4eec7f63b542 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:21:41.924129 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324ef196-82b2-4462-9192-4eec7f63b542 service nova] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:21:41.933329 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1198/2366] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:21:41 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 18 05:21:41.947124 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-7ce1e2c1-d4fb-4200-b392-6a647196b2da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d63e07595cb4ebcafb24f6928f06f61'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:21:41.947854 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-7ce1e2c1-d4fb-4200-b392-6a647196b2da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d63e07595cb4ebcafb24f6928f06f61'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:21:42.040600 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-7ce1e2c1-d4fb-4200-b392-6a647196b2da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d63e07595cb4ebcafb24f6928f06f61'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:21:42.054013 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-7ce1e2c1-d4fb-4200-b392-6a647196b2da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d63e07595cb4ebcafb24f6928f06f61'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:21:42.061558 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-7ce1e2c1-d4fb-4200-b392-6a647196b2da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d63e07595cb4ebcafb24f6928f06f61'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:21:42.067572 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1169/2367] 10.222.0.23 () {58 vars in 1223 bytes} [Sat Jun 18 05:21:41 2022] DELETE /identity/v3/users/2d63e07595cb4ebcafb24f6928f06f61 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:21:42.339081 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324ef196-82b2-4462-9192-4eec7f63b542 service nova] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:21:42.340503 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-324ef196-82b2-4462-9192-4eec7f63b542 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:21:42.354449 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324ef196-82b2-4462-9192-4eec7f63b542 service nova] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:21:42.415729 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191943bf-dbbb-4e0d-a19c-af8cc0521163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:21:42.416587 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191943bf-dbbb-4e0d-a19c-af8cc0521163 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:21:42.416866 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191943bf-dbbb-4e0d-a19c-af8cc0521163 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:21:42.417136 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191943bf-dbbb-4e0d-a19c-af8cc0521163 admin admin] PATH_INFO: `/v3/projects/fcb1e990c65747549be861c1103aafb1` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:21:42.418340 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191943bf-dbbb-4e0d-a19c-af8cc0521163 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fcb1e990c65747549be861c1103aafb1)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:21:42.420249 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191943bf-dbbb-4e0d-a19c-af8cc0521163 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:21:42.515796 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-191943bf-dbbb-4e0d-a19c-af8cc0521163 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fcb1e990c65747549be861c1103aafb1', 'request_id': 'req-191943bf-dbbb-4e0d-a19c-af8cc0521163'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:21:42.521456 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-191943bf-dbbb-4e0d-a19c-af8cc0521163 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fcb1e990c65747549be861c1103aafb1', 'request_id': 'req-191943bf-dbbb-4e0d-a19c-af8cc0521163'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:21:42.527650 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-191943bf-dbbb-4e0d-a19c-af8cc0521163 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fcb1e990c65747549be861c1103aafb1', 'request_id': 'req-191943bf-dbbb-4e0d-a19c-af8cc0521163'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:21:42.612423 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-191943bf-dbbb-4e0d-a19c-af8cc0521163 admin admin] The token cache is being invalidate because project fcb1e990c65747549be861c1103aafb1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80469) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 18 05:21:42.613050 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-191943bf-dbbb-4e0d-a19c-af8cc0521163 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:21:42.613408 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-191943bf-dbbb-4e0d-a19c-af8cc0521163 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:21:42.615027 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1199/2368] 10.222.0.23 () {58 vars in 1232 bytes} [Sat Jun 18 05:21:42 2022] DELETE /identity/v3/projects/fcb1e990c65747549be861c1103aafb1 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:21:48.727855 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24d4b19-ee80-4c17-a64d-be10be7d5868 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:21:48.728412 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24d4b19-ee80-4c17-a64d-be10be7d5868 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:21:48.728412 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24d4b19-ee80-4c17-a64d-be10be7d5868 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:21:48.747014 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-b24d4b19-ee80-4c17-a64d-be10be7d5868 None None] MFA Rules not processed for user `d2c8f964ee6a49cda501ed28a9432c8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:21:48.747570 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-b24d4b19-ee80-4c17-a64d-be10be7d5868 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:21:48.791930 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:21:48.803033 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1170/2369] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:21:48 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:21:49.480665 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191943bf-dbbb-4e0d-a19c-af8cc0521163 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:21:49.482198 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-191943bf-dbbb-4e0d-a19c-af8cc0521163 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:21:49.488130 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:21:49.558386 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191943bf-dbbb-4e0d-a19c-af8cc0521163 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:21:49.563341 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edf3a01-093f-4df8-bacf-12cc70631fc3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5323ffd91f864680b0c0084eb3f44d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:21:49.564002 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edf3a01-093f-4df8-bacf-12cc70631fc3 service placement] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:21:49.564290 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edf3a01-093f-4df8-bacf-12cc70631fc3 service placement] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:21:49.564574 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edf3a01-093f-4df8-bacf-12cc70631fc3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:21:49.566396 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-2edf3a01-093f-4df8-bacf-12cc70631fc3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:21:49.578970 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edf3a01-093f-4df8-bacf-12cc70631fc3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:21:49.580924 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edf3a01-093f-4df8-bacf-12cc70631fc3 service placement] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:21:49.591269 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1200/2370] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:21:49 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:21:52.700007 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24d4b19-ee80-4c17-a64d-be10be7d5868 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:21:52.701677 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-b24d4b19-ee80-4c17-a64d-be10be7d5868 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:21:52.783320 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24d4b19-ee80-4c17-a64d-be10be7d5868 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:21:52.787717 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93895db8-ccae-41af-9df0-bbb1675d9305 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c4d7cde8f4ad9aa479bb09e325db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:21:52.788397 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93895db8-ccae-41af-9df0-bbb1675d9305 service neutron] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:21:52.788682 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93895db8-ccae-41af-9df0-bbb1675d9305 service neutron] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:21:52.788958 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93895db8-ccae-41af-9df0-bbb1675d9305 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:21:52.790832 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-93895db8-ccae-41af-9df0-bbb1675d9305 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:21:52.800963 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93895db8-ccae-41af-9df0-bbb1675d9305 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:21:52.802442 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93895db8-ccae-41af-9df0-bbb1675d9305 service neutron] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:21:52.812977 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1171/2371] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:21:52 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:21:54.942306 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edf3a01-093f-4df8-bacf-12cc70631fc3 service placement] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:21:54.950834 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-2edf3a01-093f-4df8-bacf-12cc70631fc3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:21:55.033695 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edf3a01-093f-4df8-bacf-12cc70631fc3 service placement] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:21:55.054091 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ca6a47-4db9-4a0e-bb81-aaed6b504026 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15e7f45ee0744c1d9602187eac8e8b80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:21:55.054816 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ca6a47-4db9-4a0e-bb81-aaed6b504026 service nova] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:21:55.055102 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ca6a47-4db9-4a0e-bb81-aaed6b504026 service nova] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:21:55.055388 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ca6a47-4db9-4a0e-bb81-aaed6b504026 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:21:55.058420 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-b1ca6a47-4db9-4a0e-bb81-aaed6b504026 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:21:55.073547 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ca6a47-4db9-4a0e-bb81-aaed6b504026 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:21:55.075518 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ca6a47-4db9-4a0e-bb81-aaed6b504026 service nova] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:21:55.088975 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1201/2372] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:21:54 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 18 05:21:55.909993 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfeb2e0-b25b-4107-a9b9-23df9deb1a28 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:21:55.910507 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfeb2e0-b25b-4107-a9b9-23df9deb1a28 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:21:55.910625 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfeb2e0-b25b-4107-a9b9-23df9deb1a28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:21:55.928763 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-ecfeb2e0-b25b-4107-a9b9-23df9deb1a28 None None] MFA Rules not processed for user `216d2b492c55410d9563051142048160`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:21:55.930013 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-ecfeb2e0-b25b-4107-a9b9-23df9deb1a28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:21:55.988619 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1172/2373] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:21:55 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:22:05.082558 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=65 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:22:05.981580 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=54 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:22:06.514547 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ca6a47-4db9-4a0e-bb81-aaed6b504026 service nova] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:22:06.526192 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ca6a47-4db9-4a0e-bb81-aaed6b504026 service nova] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:22:06.531906 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88260534-a6c3-4c2c-960c-8db6f05e5018 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15e7f45ee0744c1d9602187eac8e8b80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:22:06.532532 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88260534-a6c3-4c2c-960c-8db6f05e5018 service nova] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:22:06.532812 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88260534-a6c3-4c2c-960c-8db6f05e5018 service nova] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:22:06.533084 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88260534-a6c3-4c2c-960c-8db6f05e5018 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:22:06.534834 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-88260534-a6c3-4c2c-960c-8db6f05e5018 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:22:06.592250 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88260534-a6c3-4c2c-960c-8db6f05e5018 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:22:06.593870 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88260534-a6c3-4c2c-960c-8db6f05e5018 service nova] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:22:06.602845 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1202/2374] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:22:06 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:22:16.598638 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=28 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:22:24.063951 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b83f2ad-165a-4adc-ac21-8ad93524b5a4 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:22:24.072851 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b83f2ad-165a-4adc-ac21-8ad93524b5a4 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:22:24.072851 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b83f2ad-165a-4adc-ac21-8ad93524b5a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:22:24.083167 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-7b83f2ad-165a-4adc-ac21-8ad93524b5a4 None None] MFA Rules not processed for user `e817fafd0c1b4bf9944481cf116f1260`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:22:24.085488 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-7b83f2ad-165a-4adc-ac21-8ad93524b5a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:22:24.139813 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1173/2375] 10.222.0.23 () {58 vars in 1009 bytes} [Sat Jun 18 05:22:24 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:22:24.152520 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88260534-a6c3-4c2c-960c-8db6f05e5018 service nova] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:22:24.154046 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-88260534-a6c3-4c2c-960c-8db6f05e5018 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:22:24.223681 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88260534-a6c3-4c2c-960c-8db6f05e5018 service nova] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:22:24.229311 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29063925-d01f-49ac-96b9-ae8d7448dcc7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16b49e866398400580ae48aa5a96fc04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:22:24.229987 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29063925-d01f-49ac-96b9-ae8d7448dcc7 service swift] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:22:24.230268 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29063925-d01f-49ac-96b9-ae8d7448dcc7 service swift] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:22:24.230540 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29063925-d01f-49ac-96b9-ae8d7448dcc7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:22:24.238707 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29063925-d01f-49ac-96b9-ae8d7448dcc7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:22:24.240572 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29063925-d01f-49ac-96b9-ae8d7448dcc7 service swift] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:22:24.248534 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1203/2376] 10.222.0.23 () {60 vars in 1384 bytes} [Sat Jun 18 05:22:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 18 05:22:24.434327 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b83f2ad-165a-4adc-ac21-8ad93524b5a4 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:22:24.436268 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-7b83f2ad-165a-4adc-ac21-8ad93524b5a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:22:24.456398 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b83f2ad-165a-4adc-ac21-8ad93524b5a4 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:22:24.462694 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecbf0c2-1dab-4848-b1ea-ef021c8fb7e2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15e7f45ee0744c1d9602187eac8e8b80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:22:24.463428 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecbf0c2-1dab-4848-b1ea-ef021c8fb7e2 service nova] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:22:24.463791 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecbf0c2-1dab-4848-b1ea-ef021c8fb7e2 service nova] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:22:24.464201 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecbf0c2-1dab-4848-b1ea-ef021c8fb7e2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:22:24.466210 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-9ecbf0c2-1dab-4848-b1ea-ef021c8fb7e2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:22:24.479519 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecbf0c2-1dab-4848-b1ea-ef021c8fb7e2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:22:24.481319 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecbf0c2-1dab-4848-b1ea-ef021c8fb7e2 service nova] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:22:24.497659 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1174/2377] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:22:24 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:22:32.762815 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce504d2-61aa-43e6-ae7a-fd0fddd2c6a5 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:22:32.764035 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce504d2-61aa-43e6-ae7a-fd0fddd2c6a5 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:22:32.764467 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce504d2-61aa-43e6-ae7a-fd0fddd2c6a5 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:22:32.766668 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1204/2378] 10.222.0.23 () {54 vars in 921 bytes} [Sat Jun 18 05:22:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 05:22:32.772417 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a734b6da-4e95-4ea6-b1fb-25e37c0ab7c9 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:22:32.773528 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a734b6da-4e95-4ea6-b1fb-25e37c0ab7c9 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:22:32.773528 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a734b6da-4e95-4ea6-b1fb-25e37c0ab7c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:22:32.793899 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-a734b6da-4e95-4ea6-b1fb-25e37c0ab7c9 None None] MFA Rules not processed for user `e78c4d7cde8f4ad9aa479bb09e325db3`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:22:32.795533 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-a734b6da-4e95-4ea6-b1fb-25e37c0ab7c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:22:32.809538 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1175/2379] 10.222.0.23 () {58 vars in 1018 bytes} [Sat Jun 18 05:22:32 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:22:32.821013 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce504d2-61aa-43e6-ae7a-fd0fddd2c6a5 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:22:32.824949 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-5ce504d2-61aa-43e6-ae7a-fd0fddd2c6a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:22:32.849425 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce504d2-61aa-43e6-ae7a-fd0fddd2c6a5 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:22:32.853432 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fe0ef7-d952-4b76-9eec-8975708edc69 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c4d7cde8f4ad9aa479bb09e325db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:22:32.854681 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fe0ef7-d952-4b76-9eec-8975708edc69 service neutron] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:22:32.855017 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fe0ef7-d952-4b76-9eec-8975708edc69 service neutron] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:22:32.855491 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fe0ef7-d952-4b76-9eec-8975708edc69 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:22:32.868118 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fe0ef7-d952-4b76-9eec-8975708edc69 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:22:32.870210 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fe0ef7-d952-4b76-9eec-8975708edc69 service neutron] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:22:32.880349 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1205/2380] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:22:32 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:22:42.138448 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbde9e3-8d4c-49c7-89b8-dac30379834f None None] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:22:42.138934 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbde9e3-8d4c-49c7-89b8-dac30379834f None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:22:42.139042 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbde9e3-8d4c-49c7-89b8-dac30379834f None None] PATH_INFO: `/` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:22:42.140339 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1176/2381] 10.222.0.23 () {54 vars in 921 bytes} [Sat Jun 18 05:22:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 18 05:22:42.146285 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa65be7-23e9-420d-8678-a9df48daefde None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:22:42.147073 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa65be7-23e9-420d-8678-a9df48daefde None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:22:42.147413 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa65be7-23e9-420d-8678-a9df48daefde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:22:42.188706 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-7fa65be7-23e9-420d-8678-a9df48daefde None None] MFA Rules not processed for user `15e7f45ee0744c1d9602187eac8e8b80`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:22:42.190038 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-7fa65be7-23e9-420d-8678-a9df48daefde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:22:42.200802 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1206/2382] 10.222.0.23 () {58 vars in 1018 bytes} [Sat Jun 18 05:22:42 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:22:42.212036 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbde9e3-8d4c-49c7-89b8-dac30379834f None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:22:42.222945 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbde9e3-8d4c-49c7-89b8-dac30379834f None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:22:42.227326 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a43ec29-23ae-4667-b781-c94537f7e076 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c4d7cde8f4ad9aa479bb09e325db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:22:42.227955 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a43ec29-23ae-4667-b781-c94537f7e076 service neutron] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:22:42.228241 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a43ec29-23ae-4667-b781-c94537f7e076 service neutron] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:22:42.228515 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a43ec29-23ae-4667-b781-c94537f7e076 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:22:42.239206 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a43ec29-23ae-4667-b781-c94537f7e076 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:22:42.240786 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a43ec29-23ae-4667-b781-c94537f7e076 service neutron] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:22:42.254225 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1177/2383] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:22:42 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 18 05:22:49.118281 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d29e5c-c29c-4095-b1f9-c082e730881e None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:22:49.119151 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d29e5c-c29c-4095-b1f9-c082e730881e None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:22:49.119623 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d29e5c-c29c-4095-b1f9-c082e730881e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:22:49.126045 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=48 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:22:49.140124 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-b5d29e5c-c29c-4095-b1f9-c082e730881e None None] MFA Rules not processed for user `e817fafd0c1b4bf9944481cf116f1260`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:22:49.141363 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-b5d29e5c-c29c-4095-b1f9-c082e730881e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:22:49.149955 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1207/2384] 10.222.0.23 () {58 vars in 1009 bytes} [Sat Jun 18 05:22:49 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:22:49.162057 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a43ec29-23ae-4667-b781-c94537f7e076 service neutron] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:22:49.169155 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=41 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:22:49.176261 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a43ec29-23ae-4667-b781-c94537f7e076 service neutron] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:22:49.182571 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1405cd-5fbb-43a8-b888-d3d580ed50ad service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16b49e866398400580ae48aa5a96fc04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:22:49.183296 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1405cd-5fbb-43a8-b888-d3d580ed50ad service swift] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:22:49.183653 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1405cd-5fbb-43a8-b888-d3d580ed50ad service swift] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:22:49.184034 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1405cd-5fbb-43a8-b888-d3d580ed50ad service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:22:49.193879 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1405cd-5fbb-43a8-b888-d3d580ed50ad service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:22:49.197268 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1405cd-5fbb-43a8-b888-d3d580ed50ad service swift] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:22:49.206629 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1178/2385] 10.222.0.23 () {60 vars in 1384 bytes} [Sat Jun 18 05:22:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 18 05:22:59.165488 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=2 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:22:59.354837 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=5 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:23:02.013266 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d29e5c-c29c-4095-b1f9-c082e730881e None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:23:02.014688 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-b5d29e5c-c29c-4095-b1f9-c082e730881e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:23:02.072275 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d29e5c-c29c-4095-b1f9-c082e730881e None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:23:02.076215 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59816ca-df47-4a7f-9d15-a53c0f7b595f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9de0428abc8400aa338aba4f7c34c94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:23:02.076866 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59816ca-df47-4a7f-9d15-a53c0f7b595f service cinder] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:23:02.077153 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59816ca-df47-4a7f-9d15-a53c0f7b595f service cinder] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:23:02.077464 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59816ca-df47-4a7f-9d15-a53c0f7b595f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:23:02.079275 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-a59816ca-df47-4a7f-9d15-a53c0f7b595f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:23:02.090886 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59816ca-df47-4a7f-9d15-a53c0f7b595f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:23:02.092395 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59816ca-df47-4a7f-9d15-a53c0f7b595f service cinder] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:23:02.102343 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1208/2386] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:23:02 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:23:09.033505 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03cf2d1-8dbf-430c-b9e4-c86ffc592078 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:23:09.034036 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03cf2d1-8dbf-430c-b9e4-c86ffc592078 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:23:09.034141 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03cf2d1-8dbf-430c-b9e4-c86ffc592078 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:23:09.054728 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-b03cf2d1-8dbf-430c-b9e4-c86ffc592078 None None] MFA Rules not processed for user `e817fafd0c1b4bf9944481cf116f1260`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:23:09.055997 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-b03cf2d1-8dbf-430c-b9e4-c86ffc592078 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:23:09.063446 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1179/2387] 10.222.0.23 () {58 vars in 967 bytes} [Sat Jun 18 05:23:09 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 18 05:23:09.079284 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59816ca-df47-4a7f-9d15-a53c0f7b595f service cinder] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:23:09.091480 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59816ca-df47-4a7f-9d15-a53c0f7b595f service cinder] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:23:09.096769 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6d1f1a-7b4b-4adf-b128-3f4fb4f98151 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16b49e866398400580ae48aa5a96fc04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:23:09.097607 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6d1f1a-7b4b-4adf-b128-3f4fb4f98151 service swift] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:23:09.097899 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6d1f1a-7b4b-4adf-b128-3f4fb4f98151 service swift] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:23:09.098182 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6d1f1a-7b4b-4adf-b128-3f4fb4f98151 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:23:09.108177 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6d1f1a-7b4b-4adf-b128-3f4fb4f98151 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:23:09.114370 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6d1f1a-7b4b-4adf-b128-3f4fb4f98151 service swift] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:23:09.121723 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1209/2388] 10.222.0.23 () {60 vars in 1384 bytes} [Sat Jun 18 05:23:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 18 05:23:18.275026 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03cf2d1-8dbf-430c-b9e4-c86ffc592078 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:23:18.285069 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03cf2d1-8dbf-430c-b9e4-c86ffc592078 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:23:18.289032 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb6018d-d688-4eb2-b817-190cea7ea633 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c4d7cde8f4ad9aa479bb09e325db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:23:18.289740 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb6018d-d688-4eb2-b817-190cea7ea633 service neutron] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:23:18.290022 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb6018d-d688-4eb2-b817-190cea7ea633 service neutron] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:23:18.290400 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb6018d-d688-4eb2-b817-190cea7ea633 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:23:18.292397 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-8eb6018d-d688-4eb2-b817-190cea7ea633 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:23:18.360310 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb6018d-d688-4eb2-b817-190cea7ea633 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:23:18.361792 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb6018d-d688-4eb2-b817-190cea7ea633 service neutron] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:23:18.372696 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1180/2389] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:23:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:23:19.117603 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=32 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:23:19.711896 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6cc8c6-b0c9-4580-a3e3-7e7c839fd5f5 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:23:19.712266 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6cc8c6-b0c9-4580-a3e3-7e7c839fd5f5 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:23:19.712616 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6cc8c6-b0c9-4580-a3e3-7e7c839fd5f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:23:19.731550 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-eb6cc8c6-b0c9-4580-a3e3-7e7c839fd5f5 None None] MFA Rules not processed for user `6d24c8fbdc7246e2af87004f2985a9f5`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:23:19.732888 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-eb6cc8c6-b0c9-4580-a3e3-7e7c839fd5f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:23:19.791918 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1210/2390] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:23:19 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:23:20.406700 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb6018d-d688-4eb2-b817-190cea7ea633 service neutron] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:23:20.415725 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb6018d-d688-4eb2-b817-190cea7ea633 service neutron] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:23:20.419480 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af316912-0a31-47fd-bc8b-45b7afa2e289 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c4d7cde8f4ad9aa479bb09e325db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:23:20.420122 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af316912-0a31-47fd-bc8b-45b7afa2e289 service neutron] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:23:20.420406 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af316912-0a31-47fd-bc8b-45b7afa2e289 service neutron] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:23:20.420680 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af316912-0a31-47fd-bc8b-45b7afa2e289 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:23:20.422829 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-af316912-0a31-47fd-bc8b-45b7afa2e289 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:23:20.433581 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af316912-0a31-47fd-bc8b-45b7afa2e289 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:23:20.435054 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af316912-0a31-47fd-bc8b-45b7afa2e289 service neutron] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:23:20.444112 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1181/2391] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:23:20 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:23:22.293968 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6cc8c6-b0c9-4580-a3e3-7e7c839fd5f5 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:23:22.296143 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-eb6cc8c6-b0c9-4580-a3e3-7e7c839fd5f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:23:22.312403 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6cc8c6-b0c9-4580-a3e3-7e7c839fd5f5 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:23:22.317859 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc8f1aa-749d-484f-aae6-d619a63797ef service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5323ffd91f864680b0c0084eb3f44d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:23:22.318560 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc8f1aa-749d-484f-aae6-d619a63797ef service placement] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:23:22.319235 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc8f1aa-749d-484f-aae6-d619a63797ef service placement] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:23:22.319235 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc8f1aa-749d-484f-aae6-d619a63797ef service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:23:22.325109 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-9cc8f1aa-749d-484f-aae6-d619a63797ef service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:23:22.339502 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc8f1aa-749d-484f-aae6-d619a63797ef service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:23:22.341119 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc8f1aa-749d-484f-aae6-d619a63797ef service placement] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:23:22.353917 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1211/2392] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:23:22 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:23:22.495764 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af316912-0a31-47fd-bc8b-45b7afa2e289 service neutron] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:23:22.497243 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-af316912-0a31-47fd-bc8b-45b7afa2e289 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:23:22.513607 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af316912-0a31-47fd-bc8b-45b7afa2e289 service neutron] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:23:22.519165 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa2aad9-7c72-4323-a660-4ed663730d27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:23:22.520250 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa2aad9-7c72-4323-a660-4ed663730d27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:23:22.520534 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa2aad9-7c72-4323-a660-4ed663730d27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:23:22.522765 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa2aad9-7c72-4323-a660-4ed663730d27 admin admin] PATH_INFO: `/v3/users/216d2b492c55410d9563051142048160` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:23:22.524330 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa2aad9-7c72-4323-a660-4ed663730d27 admin admin] RBAC: Authorizing `identity:delete_user(user_id=216d2b492c55410d9563051142048160)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:23:22.526297 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa2aad9-7c72-4323-a660-4ed663730d27 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:23:22.704109 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-faa2aad9-7c72-4323-a660-4ed663730d27 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '216d2b492c55410d9563051142048160'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:23:22.704758 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-faa2aad9-7c72-4323-a660-4ed663730d27 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '216d2b492c55410d9563051142048160'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:23:22.770196 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-faa2aad9-7c72-4323-a660-4ed663730d27 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '216d2b492c55410d9563051142048160'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:23:22.783508 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-faa2aad9-7c72-4323-a660-4ed663730d27 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '216d2b492c55410d9563051142048160'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:23:22.791078 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-faa2aad9-7c72-4323-a660-4ed663730d27 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '216d2b492c55410d9563051142048160'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:23:22.797068 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1182/2393] 10.222.0.23 () {58 vars in 1223 bytes} [Sat Jun 18 05:23:22 2022] DELETE /identity/v3/users/216d2b492c55410d9563051142048160 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:23:23.017799 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc8f1aa-749d-484f-aae6-d619a63797ef service placement] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:23:23.032488 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc8f1aa-749d-484f-aae6-d619a63797ef service placement] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:23:23.089933 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d12880-615d-4864-923e-737e5725ef8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:23:23.090773 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d12880-615d-4864-923e-737e5725ef8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:23:23.091060 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d12880-615d-4864-923e-737e5725ef8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:23:23.091337 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d12880-615d-4864-923e-737e5725ef8d admin admin] PATH_INFO: `/v3/projects/8820096253064e5abbe50f23bdc3ebac` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:23:23.092513 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d12880-615d-4864-923e-737e5725ef8d admin admin] RBAC: Authorizing `identity:delete_project(project_id=8820096253064e5abbe50f23bdc3ebac)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:23:23.094456 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d12880-615d-4864-923e-737e5725ef8d admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:23:23.202815 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-28d12880-615d-4864-923e-737e5725ef8d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8820096253064e5abbe50f23bdc3ebac', 'request_id': 'req-28d12880-615d-4864-923e-737e5725ef8d'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:23:23.208979 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-28d12880-615d-4864-923e-737e5725ef8d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8820096253064e5abbe50f23bdc3ebac', 'request_id': 'req-28d12880-615d-4864-923e-737e5725ef8d'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:23:23.215711 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-28d12880-615d-4864-923e-737e5725ef8d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8820096253064e5abbe50f23bdc3ebac', 'request_id': 'req-28d12880-615d-4864-923e-737e5725ef8d'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:23:23.321856 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-28d12880-615d-4864-923e-737e5725ef8d admin admin] The token cache is being invalidate because project 8820096253064e5abbe50f23bdc3ebac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80469) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 18 05:23:23.322463 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-28d12880-615d-4864-923e-737e5725ef8d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:23:23.322768 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-28d12880-615d-4864-923e-737e5725ef8d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:23:23.327864 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1212/2394] 10.222.0.23 () {58 vars in 1232 bytes} [Sat Jun 18 05:23:23 2022] DELETE /identity/v3/projects/8820096253064e5abbe50f23bdc3ebac => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:23:23.329651 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec1069a-cf2c-44f9-a376-30f6348e2f08 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:23:23.330268 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec1069a-cf2c-44f9-a376-30f6348e2f08 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:23:23.330563 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec1069a-cf2c-44f9-a376-30f6348e2f08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:23:23.353918 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-7ec1069a-cf2c-44f9-a376-30f6348e2f08 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:23:23.353918 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-7ec1069a-cf2c-44f9-a376-30f6348e2f08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:23:23.424923 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:23:23.449286 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1183/2395] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:23:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 18 05:23:23.457108 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7454a6-013e-431e-a8ce-bd126e593ba0 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:23:23.457413 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7454a6-013e-431e-a8ce-bd126e593ba0 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:23:23.457703 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7454a6-013e-431e-a8ce-bd126e593ba0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:23:23.481816 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-da7454a6-013e-431e-a8ce-bd126e593ba0 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:23:23.483127 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-da7454a6-013e-431e-a8ce-bd126e593ba0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:23:23.491696 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1213/2396] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:23:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:23:23.496010 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec1069a-cf2c-44f9-a376-30f6348e2f08 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:23:23.506711 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec1069a-cf2c-44f9-a376-30f6348e2f08 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:23:23.517138 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87f89e5-bf11-423d-9d22-a18a01477be2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:23:23.517138 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87f89e5-bf11-423d-9d22-a18a01477be2 admin admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:23:23.517138 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87f89e5-bf11-423d-9d22-a18a01477be2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:23:23.517138 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87f89e5-bf11-423d-9d22-a18a01477be2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:23:23.517509 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87f89e5-bf11-423d-9d22-a18a01477be2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 05:23:23.517509 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87f89e5-bf11-423d-9d22-a18a01477be2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:23:23.517509 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87f89e5-bf11-423d-9d22-a18a01477be2 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:23:23.528930 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1184/2397] 10.222.0.23 () {58 vars in 1152 bytes} [Sat Jun 18 05:23:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 05:23:23.533322 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7454a6-013e-431e-a8ce-bd126e593ba0 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:23:23.548159 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7454a6-013e-431e-a8ce-bd126e593ba0 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:23:23.548368 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ab4fda-5065-4f9e-8240-6113330b33c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:23:23.549022 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ab4fda-5065-4f9e-8240-6113330b33c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:23:23.549310 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ab4fda-5065-4f9e-8240-6113330b33c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:23:23.549619 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ab4fda-5065-4f9e-8240-6113330b33c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:23:23.550314 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ab4fda-5065-4f9e-8240-6113330b33c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:23:23.552205 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ab4fda-5065-4f9e-8240-6113330b33c8 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:23:23.565158 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:23:23.659623 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1214/2398] 10.222.0.23 () {60 vars in 1152 bytes} [Sat Jun 18 05:23:23 2022] POST /identity/v3/projects => generated 361 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:23:23.664525 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87f89e5-bf11-423d-9d22-a18a01477be2 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:23:23.674657 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87f89e5-bf11-423d-9d22-a18a01477be2 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:23:23.731478 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadd6fa6-a24b-4b85-b590-39c8e5ed2d86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:23:23.732196 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadd6fa6-a24b-4b85-b590-39c8e5ed2d86 admin admin] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:23:23.732476 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadd6fa6-a24b-4b85-b590-39c8e5ed2d86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:23:23.732756 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadd6fa6-a24b-4b85-b590-39c8e5ed2d86 admin admin] PATH_INFO: `/v3/users` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:23:23.733461 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadd6fa6-a24b-4b85-b590-39c8e5ed2d86 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:23:23.735366 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadd6fa6-a24b-4b85-b590-39c8e5ed2d86 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:23:23.790218 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1185/2399] 10.222.0.23 () {60 vars in 1143 bytes} [Sat Jun 18 05:23:23 2022] POST /identity/v3/users => generated 281 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:23:23.794385 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ab4fda-5065-4f9e-8240-6113330b33c8 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:23:23.805596 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ab4fda-5065-4f9e-8240-6113330b33c8 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:23:23.810202 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db893b5-33b4-49cb-b7c1-75c02e7e7902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:23:23.810876 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db893b5-33b4-49cb-b7c1-75c02e7e7902 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:23:23.811380 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db893b5-33b4-49cb-b7c1-75c02e7e7902 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:23:23.811513 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db893b5-33b4-49cb-b7c1-75c02e7e7902 admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:23:23.812181 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db893b5-33b4-49cb-b7c1-75c02e7e7902 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:23:23.813838 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db893b5-33b4-49cb-b7c1-75c02e7e7902 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:23:23.824587 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1215/2400] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:23:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:23:23.828559 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadd6fa6-a24b-4b85-b590-39c8e5ed2d86 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:23:23.839540 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadd6fa6-a24b-4b85-b590-39c8e5ed2d86 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:23:23.844265 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406c1c36-e672-41f0-b454-76ca09d275f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:23:23.844926 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406c1c36-e672-41f0-b454-76ca09d275f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:23:23.845214 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406c1c36-e672-41f0-b454-76ca09d275f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:23:23.845534 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406c1c36-e672-41f0-b454-76ca09d275f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:23:23.846295 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406c1c36-e672-41f0-b454-76ca09d275f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:23:23.847913 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406c1c36-e672-41f0-b454-76ca09d275f7 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:23:23.859265 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1186/2401] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:23:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 18 05:23:23.863542 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db893b5-33b4-49cb-b7c1-75c02e7e7902 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:23:23.873804 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db893b5-33b4-49cb-b7c1-75c02e7e7902 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:23:23.878472 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa78960-add3-4721-95d7-709af7ccda3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:23:23.879389 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa78960-add3-4721-95d7-709af7ccda3b admin admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:23:23.879675 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa78960-add3-4721-95d7-709af7ccda3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:23:23.879956 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa78960-add3-4721-95d7-709af7ccda3b admin admin] PATH_INFO: `/v3/projects/dfecb7f9d65940aa97cac89b11ebc703/users/7f6896c25db4428f9d9e3c667a28876a/roles/cbf5c4c26898413d93e73b46b41046a3` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:23:23.895853 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa78960-add3-4721-95d7-709af7ccda3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=dfecb7f9d65940aa97cac89b11ebc703, user_id=7f6896c25db4428f9d9e3c667a28876a, role_id=cbf5c4c26898413d93e73b46b41046a3)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:23:23.898818 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa78960-add3-4721-95d7-709af7ccda3b admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:23:23.955966 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1216/2402] 10.222.0.23 () {60 vars in 1482 bytes} [Sat Jun 18 05:23:23 2022] PUT /identity/v3/projects/dfecb7f9d65940aa97cac89b11ebc703/users/7f6896c25db4428f9d9e3c667a28876a/roles/cbf5c4c26898413d93e73b46b41046a3 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 18 05:23:23.966731 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae7e0e3-7845-4234-877b-082d67d62dee None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:23:23.967025 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae7e0e3-7845-4234-877b-082d67d62dee None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:23:23.967309 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae7e0e3-7845-4234-877b-082d67d62dee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:23:23.995530 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-dae7e0e3-7845-4234-877b-082d67d62dee None None] MFA Rules not processed for user `7f6896c25db4428f9d9e3c667a28876a`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:23:23.996745 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-dae7e0e3-7845-4234-877b-082d67d62dee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:23:24.063135 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1187/2403] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:23:23 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:23:24.067597 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa78960-add3-4721-95d7-709af7ccda3b admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:23:24.145683 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa78960-add3-4721-95d7-709af7ccda3b admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:23:24.152030 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978e2cb3-2be0-4bba-970a-a2670f869c86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:23:24.158102 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978e2cb3-2be0-4bba-970a-a2670f869c86 admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:23:24.158102 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978e2cb3-2be0-4bba-970a-a2670f869c86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:23:24.158102 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978e2cb3-2be0-4bba-970a-a2670f869c86 admin admin] PATH_INFO: `/v3/projects` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:23:24.158102 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978e2cb3-2be0-4bba-970a-a2670f869c86 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:23:24.158102 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978e2cb3-2be0-4bba-970a-a2670f869c86 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:23:24.278051 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1217/2404] 10.222.0.23 () {60 vars in 1152 bytes} [Sat Jun 18 05:23:24 2022] POST /identity/v3/projects => generated 365 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:23:24.281989 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae7e0e3-7845-4234-877b-082d67d62dee None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:23:24.293909 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae7e0e3-7845-4234-877b-082d67d62dee None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:23:24.340545 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:23:24.364138 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88870a5-8dfe-4481-a158-fe76a88cb627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:23:24.364825 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88870a5-8dfe-4481-a158-fe76a88cb627 admin admin] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:23:24.365093 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88870a5-8dfe-4481-a158-fe76a88cb627 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:23:24.365509 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88870a5-8dfe-4481-a158-fe76a88cb627 admin admin] PATH_INFO: `/v3/users` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:23:24.366418 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88870a5-8dfe-4481-a158-fe76a88cb627 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:23:24.368349 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88870a5-8dfe-4481-a158-fe76a88cb627 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:23:24.465261 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1188/2405] 10.222.0.23 () {60 vars in 1143 bytes} [Sat Jun 18 05:23:24 2022] POST /identity/v3/users => generated 289 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:23:24.470319 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978e2cb3-2be0-4bba-970a-a2670f869c86 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:23:24.484464 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978e2cb3-2be0-4bba-970a-a2670f869c86 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:23:24.491360 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0039b483-0008-4304-82f4-737b6c193169 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:23:24.492946 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0039b483-0008-4304-82f4-737b6c193169 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:23:24.497863 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0039b483-0008-4304-82f4-737b6c193169 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:23:24.497863 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0039b483-0008-4304-82f4-737b6c193169 admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:23:24.497863 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0039b483-0008-4304-82f4-737b6c193169 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:23:24.497863 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0039b483-0008-4304-82f4-737b6c193169 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:23:24.513717 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1218/2406] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:23:24 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:23:24.517490 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88870a5-8dfe-4481-a158-fe76a88cb627 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:23:24.537186 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88870a5-8dfe-4481-a158-fe76a88cb627 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:23:24.546839 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42691c19-a31e-4de5-b4ea-b34712357107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:23:24.547791 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42691c19-a31e-4de5-b4ea-b34712357107 admin admin] REQUEST_METHOD: `PUT` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:23:24.548085 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42691c19-a31e-4de5-b4ea-b34712357107 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:23:24.549663 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42691c19-a31e-4de5-b4ea-b34712357107 admin admin] PATH_INFO: `/v3/projects/e008d443fe5e458982fab3bf74d71ad1/users/1f0f100c3b494b66b38203bcb57b8181/roles/21c65f2c51bd4dd283d13507d2714467` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:23:24.574571 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42691c19-a31e-4de5-b4ea-b34712357107 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e008d443fe5e458982fab3bf74d71ad1, user_id=1f0f100c3b494b66b38203bcb57b8181, role_id=21c65f2c51bd4dd283d13507d2714467)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:23:24.580563 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42691c19-a31e-4de5-b4ea-b34712357107 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:23:24.677086 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1189/2407] 10.222.0.23 () {60 vars in 1482 bytes} [Sat Jun 18 05:23:24 2022] PUT /identity/v3/projects/e008d443fe5e458982fab3bf74d71ad1/users/1f0f100c3b494b66b38203bcb57b8181/roles/21c65f2c51bd4dd283d13507d2714467 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 18 05:23:24.688732 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0039b483-0008-4304-82f4-737b6c193169 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:23:24.699795 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0039b483-0008-4304-82f4-737b6c193169 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:23:24.763553 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbc0698-9508-47f9-a56d-ed6178cc0510 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:23:24.764224 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbc0698-9508-47f9-a56d-ed6178cc0510 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:23:24.764507 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbc0698-9508-47f9-a56d-ed6178cc0510 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:23:24.764784 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbc0698-9508-47f9-a56d-ed6178cc0510 admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:23:24.765540 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbc0698-9508-47f9-a56d-ed6178cc0510 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:23:24.767168 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbc0698-9508-47f9-a56d-ed6178cc0510 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:23:24.780904 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1219/2408] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:23:24 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:23:24.782170 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42691c19-a31e-4de5-b4ea-b34712357107 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:23:24.794057 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42691c19-a31e-4de5-b4ea-b34712357107 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:23:24.798923 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b472edab-980d-4244-bdb1-9cbfefc0f6bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:23:24.799826 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b472edab-980d-4244-bdb1-9cbfefc0f6bd admin admin] REQUEST_METHOD: `PUT` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:23:24.800111 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b472edab-980d-4244-bdb1-9cbfefc0f6bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:23:24.800392 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b472edab-980d-4244-bdb1-9cbfefc0f6bd admin admin] PATH_INFO: `/v3/projects/e008d443fe5e458982fab3bf74d71ad1/users/1f0f100c3b494b66b38203bcb57b8181/roles/21c65f2c51bd4dd283d13507d2714467` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:23:24.802416 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b472edab-980d-4244-bdb1-9cbfefc0f6bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=e008d443fe5e458982fab3bf74d71ad1, user_id=1f0f100c3b494b66b38203bcb57b8181, role_id=21c65f2c51bd4dd283d13507d2714467)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:23:24.805373 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b472edab-980d-4244-bdb1-9cbfefc0f6bd admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:23:24.816478 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1190/2409] 10.222.0.23 () {60 vars in 1482 bytes} [Sat Jun 18 05:23:24 2022] PUT /identity/v3/projects/e008d443fe5e458982fab3bf74d71ad1/users/1f0f100c3b494b66b38203bcb57b8181/roles/21c65f2c51bd4dd283d13507d2714467 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:23:24.828325 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842e11da-bc03-4da5-8762-7d3b7ebede38 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:23:24.828651 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842e11da-bc03-4da5-8762-7d3b7ebede38 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:23:24.828942 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842e11da-bc03-4da5-8762-7d3b7ebede38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:23:24.862036 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-842e11da-bc03-4da5-8762-7d3b7ebede38 None None] MFA Rules not processed for user `1f0f100c3b494b66b38203bcb57b8181`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:23:24.863385 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-842e11da-bc03-4da5-8762-7d3b7ebede38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:23:24.875202 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:23:24.934334 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1220/2410] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:23:24 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:23:24.945679 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b472edab-980d-4244-bdb1-9cbfefc0f6bd admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:23:24.947417 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-b472edab-980d-4244-bdb1-9cbfefc0f6bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:23:25.030630 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b472edab-980d-4244-bdb1-9cbfefc0f6bd admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:23:25.034375 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11226e4c-a02e-46ef-9e62-46fc37a53d9f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9de0428abc8400aa338aba4f7c34c94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:23:25.035046 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11226e4c-a02e-46ef-9e62-46fc37a53d9f service cinder] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:23:25.041310 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11226e4c-a02e-46ef-9e62-46fc37a53d9f service cinder] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:23:25.041310 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11226e4c-a02e-46ef-9e62-46fc37a53d9f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:23:25.051385 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11226e4c-a02e-46ef-9e62-46fc37a53d9f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:23:25.052986 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11226e4c-a02e-46ef-9e62-46fc37a53d9f service cinder] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:23:25.071975 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1191/2411] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:23:24 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:23:26.126294 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842e11da-bc03-4da5-8762-7d3b7ebede38 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:23:26.194414 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842e11da-bc03-4da5-8762-7d3b7ebede38 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:23:26.198929 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb2bef9-cd2d-4262-9b84-507cb1af3da2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:23:26.199811 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb2bef9-cd2d-4262-9b84-507cb1af3da2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:23:26.200096 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb2bef9-cd2d-4262-9b84-507cb1af3da2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:23:26.200374 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb2bef9-cd2d-4262-9b84-507cb1af3da2 admin admin] PATH_INFO: `/v3/users/7f6896c25db4428f9d9e3c667a28876a` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:23:26.201616 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb2bef9-cd2d-4262-9b84-507cb1af3da2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f6896c25db4428f9d9e3c667a28876a)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:23:26.203715 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb2bef9-cd2d-4262-9b84-507cb1af3da2 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:23:26.394014 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-1fb2bef9-cd2d-4262-9b84-507cb1af3da2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f6896c25db4428f9d9e3c667a28876a'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:23:26.407150 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-1fb2bef9-cd2d-4262-9b84-507cb1af3da2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f6896c25db4428f9d9e3c667a28876a'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:23:26.407870 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-1fb2bef9-cd2d-4262-9b84-507cb1af3da2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f6896c25db4428f9d9e3c667a28876a'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:23:26.509650 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-1fb2bef9-cd2d-4262-9b84-507cb1af3da2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f6896c25db4428f9d9e3c667a28876a'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:23:26.517365 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-1fb2bef9-cd2d-4262-9b84-507cb1af3da2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f6896c25db4428f9d9e3c667a28876a'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:23:26.524463 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1221/2412] 10.222.0.23 () {58 vars in 1223 bytes} [Sat Jun 18 05:23:26 2022] DELETE /identity/v3/users/7f6896c25db4428f9d9e3c667a28876a => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:23:26.528962 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45cbbfa-112c-4e0d-b7a1-568c281f5b82 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:23:26.529451 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45cbbfa-112c-4e0d-b7a1-568c281f5b82 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:23:26.529549 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45cbbfa-112c-4e0d-b7a1-568c281f5b82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:23:26.558303 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-a45cbbfa-112c-4e0d-b7a1-568c281f5b82 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:23:26.559686 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-a45cbbfa-112c-4e0d-b7a1-568c281f5b82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:23:26.621709 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1192/2413] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:23:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:23:26.631462 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb2bef9-cd2d-4262-9b84-507cb1af3da2 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:23:26.632879 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-1fb2bef9-cd2d-4262-9b84-507cb1af3da2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:23:26.663118 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:23:26.692169 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb2bef9-cd2d-4262-9b84-507cb1af3da2 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:23:26.696938 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430869cd-fd6f-4cc9-b5b1-6da936e92a1a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c4d7cde8f4ad9aa479bb09e325db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:23:26.697719 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430869cd-fd6f-4cc9-b5b1-6da936e92a1a service neutron] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:23:26.697996 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430869cd-fd6f-4cc9-b5b1-6da936e92a1a service neutron] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:23:26.698267 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430869cd-fd6f-4cc9-b5b1-6da936e92a1a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:23:26.707539 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430869cd-fd6f-4cc9-b5b1-6da936e92a1a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:23:26.709497 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430869cd-fd6f-4cc9-b5b1-6da936e92a1a service neutron] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:23:26.719945 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1222/2414] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:23:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:23:27.204580 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45cbbfa-112c-4e0d-b7a1-568c281f5b82 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:23:27.206216 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-a45cbbfa-112c-4e0d-b7a1-568c281f5b82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:23:27.219497 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45cbbfa-112c-4e0d-b7a1-568c281f5b82 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:23:27.224568 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf022af-0b18-47e1-9239-e3bbf6c3e298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:23:27.225527 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf022af-0b18-47e1-9239-e3bbf6c3e298 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:23:27.225820 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf022af-0b18-47e1-9239-e3bbf6c3e298 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:23:27.226269 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf022af-0b18-47e1-9239-e3bbf6c3e298 admin admin] PATH_INFO: `/v3/projects/dfecb7f9d65940aa97cac89b11ebc703` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:23:27.227837 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf022af-0b18-47e1-9239-e3bbf6c3e298 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dfecb7f9d65940aa97cac89b11ebc703)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:23:27.229747 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf022af-0b18-47e1-9239-e3bbf6c3e298 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:23:27.338903 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:23:27.351758 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-bcf022af-0b18-47e1-9239-e3bbf6c3e298 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dfecb7f9d65940aa97cac89b11ebc703', 'request_id': 'req-bcf022af-0b18-47e1-9239-e3bbf6c3e298'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:23:27.357241 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-bcf022af-0b18-47e1-9239-e3bbf6c3e298 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dfecb7f9d65940aa97cac89b11ebc703', 'request_id': 'req-bcf022af-0b18-47e1-9239-e3bbf6c3e298'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:23:27.476249 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-bcf022af-0b18-47e1-9239-e3bbf6c3e298 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dfecb7f9d65940aa97cac89b11ebc703', 'request_id': 'req-bcf022af-0b18-47e1-9239-e3bbf6c3e298'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:23:27.490657 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-bcf022af-0b18-47e1-9239-e3bbf6c3e298 admin admin] The token cache is being invalidate because project dfecb7f9d65940aa97cac89b11ebc703 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80468) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 18 05:23:27.499520 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-bcf022af-0b18-47e1-9239-e3bbf6c3e298 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:23:27.499894 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-bcf022af-0b18-47e1-9239-e3bbf6c3e298 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:23:27.501834 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1193/2415] 10.222.0.23 () {58 vars in 1232 bytes} [Sat Jun 18 05:23:27 2022] DELETE /identity/v3/projects/dfecb7f9d65940aa97cac89b11ebc703 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:23:27.505667 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430869cd-fd6f-4cc9-b5b1-6da936e92a1a service neutron] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:23:27.516742 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430869cd-fd6f-4cc9-b5b1-6da936e92a1a service neutron] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:23:27.580492 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36852a0f-0844-4909-a9f9-5340c1a4cb5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:23:27.581514 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36852a0f-0844-4909-a9f9-5340c1a4cb5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:23:27.585882 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36852a0f-0844-4909-a9f9-5340c1a4cb5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:23:27.585882 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36852a0f-0844-4909-a9f9-5340c1a4cb5a admin admin] PATH_INFO: `/v3/users/1f0f100c3b494b66b38203bcb57b8181` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:23:27.585882 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36852a0f-0844-4909-a9f9-5340c1a4cb5a admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f0f100c3b494b66b38203bcb57b8181)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:23:27.586435 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36852a0f-0844-4909-a9f9-5340c1a4cb5a admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:23:27.767308 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-36852a0f-0844-4909-a9f9-5340c1a4cb5a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f0f100c3b494b66b38203bcb57b8181'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:23:27.780639 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-36852a0f-0844-4909-a9f9-5340c1a4cb5a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f0f100c3b494b66b38203bcb57b8181'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:23:27.781464 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-36852a0f-0844-4909-a9f9-5340c1a4cb5a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f0f100c3b494b66b38203bcb57b8181'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:23:27.846092 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-36852a0f-0844-4909-a9f9-5340c1a4cb5a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f0f100c3b494b66b38203bcb57b8181'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:23:27.853361 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-36852a0f-0844-4909-a9f9-5340c1a4cb5a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f0f100c3b494b66b38203bcb57b8181'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:23:27.859405 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1223/2416] 10.222.0.23 () {58 vars in 1223 bytes} [Sat Jun 18 05:23:27 2022] DELETE /identity/v3/users/1f0f100c3b494b66b38203bcb57b8181 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:23:28.329181 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf022af-0b18-47e1-9239-e3bbf6c3e298 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:23:28.330673 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-bcf022af-0b18-47e1-9239-e3bbf6c3e298 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:23:28.346043 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf022af-0b18-47e1-9239-e3bbf6c3e298 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:23:28.404047 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99866c9e-7442-49f4-8e3a-7f1a9cbfb1c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:23:28.405162 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99866c9e-7442-49f4-8e3a-7f1a9cbfb1c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:23:28.405467 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99866c9e-7442-49f4-8e3a-7f1a9cbfb1c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:23:28.405743 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99866c9e-7442-49f4-8e3a-7f1a9cbfb1c0 admin admin] PATH_INFO: `/v3/projects/e008d443fe5e458982fab3bf74d71ad1` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:23:28.406953 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99866c9e-7442-49f4-8e3a-7f1a9cbfb1c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e008d443fe5e458982fab3bf74d71ad1)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:23:28.408880 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99866c9e-7442-49f4-8e3a-7f1a9cbfb1c0 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:23:28.529309 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-99866c9e-7442-49f4-8e3a-7f1a9cbfb1c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e008d443fe5e458982fab3bf74d71ad1', 'request_id': 'req-99866c9e-7442-49f4-8e3a-7f1a9cbfb1c0'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:23:28.535670 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-99866c9e-7442-49f4-8e3a-7f1a9cbfb1c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e008d443fe5e458982fab3bf74d71ad1', 'request_id': 'req-99866c9e-7442-49f4-8e3a-7f1a9cbfb1c0'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:23:28.606585 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-99866c9e-7442-49f4-8e3a-7f1a9cbfb1c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e008d443fe5e458982fab3bf74d71ad1', 'request_id': 'req-99866c9e-7442-49f4-8e3a-7f1a9cbfb1c0'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:23:28.619337 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-99866c9e-7442-49f4-8e3a-7f1a9cbfb1c0 admin admin] The token cache is being invalidate because project e008d443fe5e458982fab3bf74d71ad1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80468) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 18 05:23:28.619627 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-99866c9e-7442-49f4-8e3a-7f1a9cbfb1c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:23:28.620262 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-99866c9e-7442-49f4-8e3a-7f1a9cbfb1c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:23:28.621830 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1194/2417] 10.222.0.23 () {58 vars in 1232 bytes} [Sat Jun 18 05:23:28 2022] DELETE /identity/v3/projects/e008d443fe5e458982fab3bf74d71ad1 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:23:28.626689 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ebe10b-79bf-4cec-9a31-6aa4557b22f1 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:23:28.627172 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ebe10b-79bf-4cec-9a31-6aa4557b22f1 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:23:28.627295 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ebe10b-79bf-4cec-9a31-6aa4557b22f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:23:28.645174 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-00ebe10b-79bf-4cec-9a31-6aa4557b22f1 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:23:28.648040 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-00ebe10b-79bf-4cec-9a31-6aa4557b22f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:23:28.689565 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:23:28.707331 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1224/2418] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:23:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:23:28.715529 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48552315-bb22-4361-84e0-0b8cd6e87b10 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:23:28.715834 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48552315-bb22-4361-84e0-0b8cd6e87b10 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:23:28.716099 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48552315-bb22-4361-84e0-0b8cd6e87b10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:23:28.735250 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-48552315-bb22-4361-84e0-0b8cd6e87b10 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:23:28.736458 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-48552315-bb22-4361-84e0-0b8cd6e87b10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:23:28.745965 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1195/2419] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:23:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:23:28.750245 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ebe10b-79bf-4cec-9a31-6aa4557b22f1 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:23:28.760399 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ebe10b-79bf-4cec-9a31-6aa4557b22f1 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:23:28.765345 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf6e12f-2672-4812-952f-302c98306e81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:23:28.765843 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf6e12f-2672-4812-952f-302c98306e81 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:23:28.766124 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf6e12f-2672-4812-952f-302c98306e81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:23:28.766399 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf6e12f-2672-4812-952f-302c98306e81 admin admin] PATH_INFO: `/v3/domains` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:23:28.767118 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf6e12f-2672-4812-952f-302c98306e81 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 05:23:28.767427 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf6e12f-2672-4812-952f-302c98306e81 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:23:28.769036 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf6e12f-2672-4812-952f-302c98306e81 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:23:28.782183 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1225/2420] 10.222.0.23 () {58 vars in 1152 bytes} [Sat Jun 18 05:23:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 05:23:28.786402 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48552315-bb22-4361-84e0-0b8cd6e87b10 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:23:28.796515 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48552315-bb22-4361-84e0-0b8cd6e87b10 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:23:28.801275 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14734bb4-009f-4430-8c87-15fe8e188561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:23:28.801975 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14734bb4-009f-4430-8c87-15fe8e188561 admin admin] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:23:28.802235 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14734bb4-009f-4430-8c87-15fe8e188561 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:23:28.802507 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14734bb4-009f-4430-8c87-15fe8e188561 admin admin] PATH_INFO: `/v3/projects` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:23:28.803266 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14734bb4-009f-4430-8c87-15fe8e188561 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:23:28.805217 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14734bb4-009f-4430-8c87-15fe8e188561 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:23:28.891399 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1196/2421] 10.222.0.23 () {60 vars in 1152 bytes} [Sat Jun 18 05:23:28 2022] POST /identity/v3/projects => generated 363 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:23:28.895598 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf6e12f-2672-4812-952f-302c98306e81 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:23:28.907604 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf6e12f-2672-4812-952f-302c98306e81 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:23:28.964036 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d87601c-44dd-4a45-9c9a-c82bdc10a180 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:23:28.964692 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d87601c-44dd-4a45-9c9a-c82bdc10a180 admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:23:28.964977 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d87601c-44dd-4a45-9c9a-c82bdc10a180 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:23:28.965254 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d87601c-44dd-4a45-9c9a-c82bdc10a180 admin admin] PATH_INFO: `/v3/users` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:23:28.965982 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d87601c-44dd-4a45-9c9a-c82bdc10a180 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:23:28.967910 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d87601c-44dd-4a45-9c9a-c82bdc10a180 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:23:29.050485 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1226/2422] 10.222.0.23 () {60 vars in 1143 bytes} [Sat Jun 18 05:23:28 2022] POST /identity/v3/users => generated 282 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 18 05:23:29.055062 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14734bb4-009f-4430-8c87-15fe8e188561 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:23:29.065374 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14734bb4-009f-4430-8c87-15fe8e188561 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:23:29.070419 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb30c26-73a3-4bad-a060-38c6e99ed188 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:23:29.071066 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb30c26-73a3-4bad-a060-38c6e99ed188 admin admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:23:29.071380 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb30c26-73a3-4bad-a060-38c6e99ed188 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:23:29.071659 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb30c26-73a3-4bad-a060-38c6e99ed188 admin admin] PATH_INFO: `/v3/roles` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:23:29.072382 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb30c26-73a3-4bad-a060-38c6e99ed188 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:23:29.074002 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb30c26-73a3-4bad-a060-38c6e99ed188 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:23:29.084866 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1197/2423] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:23:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:23:29.088820 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d87601c-44dd-4a45-9c9a-c82bdc10a180 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:23:29.098736 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d87601c-44dd-4a45-9c9a-c82bdc10a180 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:23:29.104497 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cfbd23-7013-423f-b304-aa2be828f9e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:23:29.105142 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cfbd23-7013-423f-b304-aa2be828f9e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:23:29.105470 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cfbd23-7013-423f-b304-aa2be828f9e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:23:29.105753 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cfbd23-7013-423f-b304-aa2be828f9e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:23:29.106481 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cfbd23-7013-423f-b304-aa2be828f9e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:23:29.108119 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cfbd23-7013-423f-b304-aa2be828f9e8 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:23:29.118604 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1227/2424] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:23:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 18 05:23:29.122508 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb30c26-73a3-4bad-a060-38c6e99ed188 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:23:29.132975 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb30c26-73a3-4bad-a060-38c6e99ed188 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:23:29.137829 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e71a79c-1987-4420-98ad-888488f868c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:23:29.138703 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e71a79c-1987-4420-98ad-888488f868c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:23:29.138988 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e71a79c-1987-4420-98ad-888488f868c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:23:29.139269 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e71a79c-1987-4420-98ad-888488f868c3 admin admin] PATH_INFO: `/v3/projects/40bb47fea5624d43b18743e2eee9ad5d/users/61a32a3143f840428677f1e05e91aae6/roles/cbf5c4c26898413d93e73b46b41046a3` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:23:29.148006 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:23:29.155118 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e71a79c-1987-4420-98ad-888488f868c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=40bb47fea5624d43b18743e2eee9ad5d, user_id=61a32a3143f840428677f1e05e91aae6, role_id=cbf5c4c26898413d93e73b46b41046a3)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:23:29.157815 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e71a79c-1987-4420-98ad-888488f868c3 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:23:29.232980 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1198/2425] 10.222.0.23 () {60 vars in 1482 bytes} [Sat Jun 18 05:23:29 2022] PUT /identity/v3/projects/40bb47fea5624d43b18743e2eee9ad5d/users/61a32a3143f840428677f1e05e91aae6/roles/cbf5c4c26898413d93e73b46b41046a3 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:23:29.244617 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87343478-e9be-42d9-8730-5568c9dc82c2 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:23:29.244908 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87343478-e9be-42d9-8730-5568c9dc82c2 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:23:29.245189 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87343478-e9be-42d9-8730-5568c9dc82c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:23:29.272622 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-87343478-e9be-42d9-8730-5568c9dc82c2 None None] MFA Rules not processed for user `61a32a3143f840428677f1e05e91aae6`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:23:29.273971 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-87343478-e9be-42d9-8730-5568c9dc82c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:23:29.340168 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1228/2426] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:23:29 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:23:29.344330 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e71a79c-1987-4420-98ad-888488f868c3 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:23:29.415733 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e71a79c-1987-4420-98ad-888488f868c3 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:23:29.421564 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecca23c2-2174-4295-9a4b-6b83e5dec295 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:23:29.422219 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecca23c2-2174-4295-9a4b-6b83e5dec295 admin admin] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:23:29.422504 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecca23c2-2174-4295-9a4b-6b83e5dec295 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:23:29.422782 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecca23c2-2174-4295-9a4b-6b83e5dec295 admin admin] PATH_INFO: `/v3/projects` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:23:29.423514 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecca23c2-2174-4295-9a4b-6b83e5dec295 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:23:29.425447 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecca23c2-2174-4295-9a4b-6b83e5dec295 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:23:29.507664 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1199/2427] 10.222.0.23 () {60 vars in 1152 bytes} [Sat Jun 18 05:23:29 2022] POST /identity/v3/projects => generated 365 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 18 05:23:29.515483 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87343478-e9be-42d9-8730-5568c9dc82c2 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:23:29.526203 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87343478-e9be-42d9-8730-5568c9dc82c2 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:23:29.568229 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:23:29.585777 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5ccddb-e613-4d01-84ab-d02fa3840eac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:23:29.586456 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5ccddb-e613-4d01-84ab-d02fa3840eac admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:23:29.586744 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5ccddb-e613-4d01-84ab-d02fa3840eac admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:23:29.587023 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5ccddb-e613-4d01-84ab-d02fa3840eac admin admin] PATH_INFO: `/v3/users` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:23:29.587721 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5ccddb-e613-4d01-84ab-d02fa3840eac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:23:29.589762 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5ccddb-e613-4d01-84ab-d02fa3840eac admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:23:29.712600 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1229/2428] 10.222.0.23 () {60 vars in 1143 bytes} [Sat Jun 18 05:23:29 2022] POST /identity/v3/users => generated 289 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:23:29.714297 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecca23c2-2174-4295-9a4b-6b83e5dec295 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:23:29.725500 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecca23c2-2174-4295-9a4b-6b83e5dec295 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:23:29.729988 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e126f1a-45f6-487a-b8b1-b76092a753fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:23:29.730696 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e126f1a-45f6-487a-b8b1-b76092a753fe admin admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:23:29.730988 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e126f1a-45f6-487a-b8b1-b76092a753fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:23:29.731264 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e126f1a-45f6-487a-b8b1-b76092a753fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:23:29.731993 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e126f1a-45f6-487a-b8b1-b76092a753fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:23:29.733606 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e126f1a-45f6-487a-b8b1-b76092a753fe admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:23:29.745257 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1200/2429] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:23:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:23:29.749405 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5ccddb-e613-4d01-84ab-d02fa3840eac admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:23:29.760665 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5ccddb-e613-4d01-84ab-d02fa3840eac admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:23:29.765261 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a6cd59-745d-44be-b5c4-74240782c99e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:23:29.766180 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a6cd59-745d-44be-b5c4-74240782c99e admin admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:23:29.766458 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a6cd59-745d-44be-b5c4-74240782c99e admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:23:29.766731 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a6cd59-745d-44be-b5c4-74240782c99e admin admin] PATH_INFO: `/v3/projects/ed03a00e74a44076951809ea8bc94407/users/51158ce3de4344cabdd3dae5215a28b8/roles/21c65f2c51bd4dd283d13507d2714467` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:23:29.784539 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a6cd59-745d-44be-b5c4-74240782c99e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed03a00e74a44076951809ea8bc94407, user_id=51158ce3de4344cabdd3dae5215a28b8, role_id=21c65f2c51bd4dd283d13507d2714467)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:23:29.792835 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a6cd59-745d-44be-b5c4-74240782c99e admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:23:29.881498 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1230/2430] 10.222.0.23 () {60 vars in 1482 bytes} [Sat Jun 18 05:23:29 2022] PUT /identity/v3/projects/ed03a00e74a44076951809ea8bc94407/users/51158ce3de4344cabdd3dae5215a28b8/roles/21c65f2c51bd4dd283d13507d2714467 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:23:29.885172 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e126f1a-45f6-487a-b8b1-b76092a753fe admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:23:29.895154 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e126f1a-45f6-487a-b8b1-b76092a753fe admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:23:29.955181 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b07ddc-4c86-4bab-b27f-83fb7f0e6b4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:23:29.955848 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b07ddc-4c86-4bab-b27f-83fb7f0e6b4b admin admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:23:29.956136 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b07ddc-4c86-4bab-b27f-83fb7f0e6b4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:23:29.956412 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b07ddc-4c86-4bab-b27f-83fb7f0e6b4b admin admin] PATH_INFO: `/v3/roles` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:23:29.957138 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b07ddc-4c86-4bab-b27f-83fb7f0e6b4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:23:29.958764 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b07ddc-4c86-4bab-b27f-83fb7f0e6b4b admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:23:29.971556 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1201/2431] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:23:29 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:23:29.972751 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a6cd59-745d-44be-b5c4-74240782c99e admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:23:29.983800 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a6cd59-745d-44be-b5c4-74240782c99e admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:23:29.988610 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c1ece8-e482-4fc5-81b0-4262de70b556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:23:29.989514 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c1ece8-e482-4fc5-81b0-4262de70b556 admin admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:23:29.989802 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c1ece8-e482-4fc5-81b0-4262de70b556 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:23:29.990097 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c1ece8-e482-4fc5-81b0-4262de70b556 admin admin] PATH_INFO: `/v3/projects/ed03a00e74a44076951809ea8bc94407/users/51158ce3de4344cabdd3dae5215a28b8/roles/21c65f2c51bd4dd283d13507d2714467` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:23:29.992489 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c1ece8-e482-4fc5-81b0-4262de70b556 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed03a00e74a44076951809ea8bc94407, user_id=51158ce3de4344cabdd3dae5215a28b8, role_id=21c65f2c51bd4dd283d13507d2714467)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:23:29.995178 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c1ece8-e482-4fc5-81b0-4262de70b556 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:23:30.005708 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1231/2432] 10.222.0.23 () {60 vars in 1482 bytes} [Sat Jun 18 05:23:29 2022] PUT /identity/v3/projects/ed03a00e74a44076951809ea8bc94407/users/51158ce3de4344cabdd3dae5215a28b8/roles/21c65f2c51bd4dd283d13507d2714467 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:23:30.016339 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eb5c83-0de4-4100-b3c9-fcab1e114ee9 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:23:30.016703 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eb5c83-0de4-4100-b3c9-fcab1e114ee9 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:23:30.016991 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eb5c83-0de4-4100-b3c9-fcab1e114ee9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:23:30.046106 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-17eb5c83-0de4-4100-b3c9-fcab1e114ee9 None None] MFA Rules not processed for user `51158ce3de4344cabdd3dae5215a28b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:23:30.047885 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-17eb5c83-0de4-4100-b3c9-fcab1e114ee9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:23:30.092970 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:23:30.134575 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1202/2433] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:23:30 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:23:30.140721 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c1ece8-e482-4fc5-81b0-4262de70b556 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:23:30.142559 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-27c1ece8-e482-4fc5-81b0-4262de70b556 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:23:30.215367 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c1ece8-e482-4fc5-81b0-4262de70b556 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:23:30.219870 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5b2d36-91d5-4a5e-baa3-698fffa1de59 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9de0428abc8400aa338aba4f7c34c94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:23:30.245057 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5b2d36-91d5-4a5e-baa3-698fffa1de59 service cinder] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:23:30.245057 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5b2d36-91d5-4a5e-baa3-698fffa1de59 service cinder] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:23:30.245057 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5b2d36-91d5-4a5e-baa3-698fffa1de59 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:23:30.245057 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5b2d36-91d5-4a5e-baa3-698fffa1de59 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:23:30.245057 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5b2d36-91d5-4a5e-baa3-698fffa1de59 service cinder] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:23:30.245057 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1232/2434] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:23:30 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:23:30.622058 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17eb5c83-0de4-4100-b3c9-fcab1e114ee9 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:23:30.631708 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17eb5c83-0de4-4100-b3c9-fcab1e114ee9 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:23:30.635473 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71921f5d-231a-427d-bbb2-8f65aee6d6b5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9de0428abc8400aa338aba4f7c34c94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:23:30.636115 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71921f5d-231a-427d-bbb2-8f65aee6d6b5 service cinder] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:23:30.636396 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71921f5d-231a-427d-bbb2-8f65aee6d6b5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:23:30.636668 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71921f5d-231a-427d-bbb2-8f65aee6d6b5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:23:30.700690 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71921f5d-231a-427d-bbb2-8f65aee6d6b5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:23:30.702241 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71921f5d-231a-427d-bbb2-8f65aee6d6b5 service cinder] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:23:30.711525 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1203/2435] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:23:30 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:23:34.303736 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5b2d36-91d5-4a5e-baa3-698fffa1de59 service cinder] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:23:34.391195 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5b2d36-91d5-4a5e-baa3-698fffa1de59 service cinder] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:23:34.397613 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c280c3-b15e-49ae-bcf8-533d0fec7d62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:23:34.398487 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c280c3-b15e-49ae-bcf8-533d0fec7d62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:23:34.398776 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c280c3-b15e-49ae-bcf8-533d0fec7d62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:23:34.399057 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c280c3-b15e-49ae-bcf8-533d0fec7d62 admin admin] PATH_INFO: `/v3/users/61a32a3143f840428677f1e05e91aae6` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:23:34.400276 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c280c3-b15e-49ae-bcf8-533d0fec7d62 admin admin] RBAC: Authorizing `identity:delete_user(user_id=61a32a3143f840428677f1e05e91aae6)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:23:34.402257 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c280c3-b15e-49ae-bcf8-533d0fec7d62 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:23:34.590701 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:23:34.597905 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-c7c280c3-b15e-49ae-bcf8-533d0fec7d62 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61a32a3143f840428677f1e05e91aae6'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:23:34.609631 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-c7c280c3-b15e-49ae-bcf8-533d0fec7d62 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61a32a3143f840428677f1e05e91aae6'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:23:34.610335 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-c7c280c3-b15e-49ae-bcf8-533d0fec7d62 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61a32a3143f840428677f1e05e91aae6'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:23:34.703667 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-c7c280c3-b15e-49ae-bcf8-533d0fec7d62 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61a32a3143f840428677f1e05e91aae6'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:23:34.711830 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-c7c280c3-b15e-49ae-bcf8-533d0fec7d62 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61a32a3143f840428677f1e05e91aae6'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:23:34.727796 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1233/2436] 10.222.0.23 () {58 vars in 1223 bytes} [Sat Jun 18 05:23:34 2022] DELETE /identity/v3/users/61a32a3143f840428677f1e05e91aae6 => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:23:34.730998 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec400cf-ad06-453e-9f9e-b5ee8d8392d9 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:23:34.731474 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec400cf-ad06-453e-9f9e-b5ee8d8392d9 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:23:34.731565 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec400cf-ad06-453e-9f9e-b5ee8d8392d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:23:34.752158 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-eec400cf-ad06-453e-9f9e-b5ee8d8392d9 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:23:34.759854 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-eec400cf-ad06-453e-9f9e-b5ee8d8392d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:23:34.827752 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1204/2437] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:23:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:23:34.838149 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c280c3-b15e-49ae-bcf8-533d0fec7d62 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:23:34.839718 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-c7c280c3-b15e-49ae-bcf8-533d0fec7d62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:23:34.923970 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c280c3-b15e-49ae-bcf8-533d0fec7d62 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:23:34.928694 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b7763c-4e8f-4b56-9427-1a5e377e69f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c4d7cde8f4ad9aa479bb09e325db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:23:34.929486 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b7763c-4e8f-4b56-9427-1a5e377e69f3 service neutron] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:23:34.929880 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b7763c-4e8f-4b56-9427-1a5e377e69f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:23:34.930252 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b7763c-4e8f-4b56-9427-1a5e377e69f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:23:34.945950 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b7763c-4e8f-4b56-9427-1a5e377e69f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:23:34.947388 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b7763c-4e8f-4b56-9427-1a5e377e69f3 service neutron] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:23:34.963168 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1234/2438] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:23:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:23:35.453059 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec400cf-ad06-453e-9f9e-b5ee8d8392d9 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:23:35.455069 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-eec400cf-ad06-453e-9f9e-b5ee8d8392d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:23:35.476471 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec400cf-ad06-453e-9f9e-b5ee8d8392d9 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:23:35.481510 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ef1d57-d4ea-4f44-9ff5-7788bc4d179a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:23:35.482902 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ef1d57-d4ea-4f44-9ff5-7788bc4d179a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:23:35.483192 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ef1d57-d4ea-4f44-9ff5-7788bc4d179a admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:23:35.483493 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ef1d57-d4ea-4f44-9ff5-7788bc4d179a admin admin] PATH_INFO: `/v3/projects/40bb47fea5624d43b18743e2eee9ad5d` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:23:35.484714 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ef1d57-d4ea-4f44-9ff5-7788bc4d179a admin admin] RBAC: Authorizing `identity:delete_project(project_id=40bb47fea5624d43b18743e2eee9ad5d)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:23:35.486728 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ef1d57-d4ea-4f44-9ff5-7788bc4d179a admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:23:35.583884 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-79ef1d57-d4ea-4f44-9ff5-7788bc4d179a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40bb47fea5624d43b18743e2eee9ad5d', 'request_id': 'req-79ef1d57-d4ea-4f44-9ff5-7788bc4d179a'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:23:35.589616 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-79ef1d57-d4ea-4f44-9ff5-7788bc4d179a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40bb47fea5624d43b18743e2eee9ad5d', 'request_id': 'req-79ef1d57-d4ea-4f44-9ff5-7788bc4d179a'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:23:35.673128 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-79ef1d57-d4ea-4f44-9ff5-7788bc4d179a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40bb47fea5624d43b18743e2eee9ad5d', 'request_id': 'req-79ef1d57-d4ea-4f44-9ff5-7788bc4d179a'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:23:35.685456 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-79ef1d57-d4ea-4f44-9ff5-7788bc4d179a admin admin] The token cache is being invalidate because project 40bb47fea5624d43b18743e2eee9ad5d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80468) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 18 05:23:35.685744 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-79ef1d57-d4ea-4f44-9ff5-7788bc4d179a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:23:35.686503 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-79ef1d57-d4ea-4f44-9ff5-7788bc4d179a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:23:35.688035 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1205/2439] 10.222.0.23 () {58 vars in 1232 bytes} [Sat Jun 18 05:23:35 2022] DELETE /identity/v3/projects/40bb47fea5624d43b18743e2eee9ad5d => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:23:35.692256 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b7763c-4e8f-4b56-9427-1a5e377e69f3 service neutron] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:23:35.702995 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b7763c-4e8f-4b56-9427-1a5e377e69f3 service neutron] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:23:35.758532 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a56651d-7d34-41bc-a885-f2684f0b08c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:23:35.759402 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a56651d-7d34-41bc-a885-f2684f0b08c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:23:35.759685 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a56651d-7d34-41bc-a885-f2684f0b08c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:23:35.759961 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a56651d-7d34-41bc-a885-f2684f0b08c4 admin admin] PATH_INFO: `/v3/users/51158ce3de4344cabdd3dae5215a28b8` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:23:35.761146 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a56651d-7d34-41bc-a885-f2684f0b08c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=51158ce3de4344cabdd3dae5215a28b8)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:23:35.763142 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a56651d-7d34-41bc-a885-f2684f0b08c4 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:23:35.973769 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-9a56651d-7d34-41bc-a885-f2684f0b08c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51158ce3de4344cabdd3dae5215a28b8'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:23:35.984786 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:23:35.986999 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-9a56651d-7d34-41bc-a885-f2684f0b08c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51158ce3de4344cabdd3dae5215a28b8'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:23:35.987656 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-9a56651d-7d34-41bc-a885-f2684f0b08c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51158ce3de4344cabdd3dae5215a28b8'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:23:36.074325 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-9a56651d-7d34-41bc-a885-f2684f0b08c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51158ce3de4344cabdd3dae5215a28b8'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:23:36.081672 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-9a56651d-7d34-41bc-a885-f2684f0b08c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51158ce3de4344cabdd3dae5215a28b8'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:23:36.088160 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1235/2440] 10.222.0.23 () {58 vars in 1223 bytes} [Sat Jun 18 05:23:35 2022] DELETE /identity/v3/users/51158ce3de4344cabdd3dae5215a28b8 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:23:36.513351 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ef1d57-d4ea-4f44-9ff5-7788bc4d179a admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:23:36.515569 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-79ef1d57-d4ea-4f44-9ff5-7788bc4d179a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:23:36.527987 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:23:36.533552 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ef1d57-d4ea-4f44-9ff5-7788bc4d179a admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:23:36.604278 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad57e07-310b-4a4a-aebe-a4498bbee69a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:23:36.605154 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad57e07-310b-4a4a-aebe-a4498bbee69a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:23:36.605446 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad57e07-310b-4a4a-aebe-a4498bbee69a admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:23:36.605726 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad57e07-310b-4a4a-aebe-a4498bbee69a admin admin] PATH_INFO: `/v3/projects/ed03a00e74a44076951809ea8bc94407` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:23:36.613042 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad57e07-310b-4a4a-aebe-a4498bbee69a admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed03a00e74a44076951809ea8bc94407)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:23:36.613042 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad57e07-310b-4a4a-aebe-a4498bbee69a admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:23:36.719520 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-cad57e07-310b-4a4a-aebe-a4498bbee69a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed03a00e74a44076951809ea8bc94407', 'request_id': 'req-cad57e07-310b-4a4a-aebe-a4498bbee69a'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:23:36.725739 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-cad57e07-310b-4a4a-aebe-a4498bbee69a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed03a00e74a44076951809ea8bc94407', 'request_id': 'req-cad57e07-310b-4a4a-aebe-a4498bbee69a'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:23:36.839063 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-cad57e07-310b-4a4a-aebe-a4498bbee69a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed03a00e74a44076951809ea8bc94407', 'request_id': 'req-cad57e07-310b-4a4a-aebe-a4498bbee69a'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:23:36.854331 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-cad57e07-310b-4a4a-aebe-a4498bbee69a admin admin] The token cache is being invalidate because project ed03a00e74a44076951809ea8bc94407 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80468) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 18 05:23:36.855044 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-cad57e07-310b-4a4a-aebe-a4498bbee69a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:23:36.856300 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-cad57e07-310b-4a4a-aebe-a4498bbee69a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:23:36.858180 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1206/2441] 10.222.0.23 () {58 vars in 1232 bytes} [Sat Jun 18 05:23:36 2022] DELETE /identity/v3/projects/ed03a00e74a44076951809ea8bc94407 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:23:36.863001 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714b485d-2c11-4a32-a61d-ab5b3ba2b4db None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:23:36.863525 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714b485d-2c11-4a32-a61d-ab5b3ba2b4db None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:23:36.863765 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714b485d-2c11-4a32-a61d-ab5b3ba2b4db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:23:36.887762 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-714b485d-2c11-4a32-a61d-ab5b3ba2b4db None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:23:36.889002 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-714b485d-2c11-4a32-a61d-ab5b3ba2b4db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:23:36.953630 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1236/2442] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:23:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:23:36.961271 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa284da-d246-4dad-be0c-d1da412dc0a3 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:23:36.961595 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa284da-d246-4dad-be0c-d1da412dc0a3 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:23:36.961904 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa284da-d246-4dad-be0c-d1da412dc0a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:23:36.984783 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-9aa284da-d246-4dad-be0c-d1da412dc0a3 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:23:36.986270 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-9aa284da-d246-4dad-be0c-d1da412dc0a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:23:36.999541 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714b485d-2c11-4a32-a61d-ab5b3ba2b4db None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:23:37.004355 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1207/2443] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:23:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:23:37.010201 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714b485d-2c11-4a32-a61d-ab5b3ba2b4db None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:23:37.015247 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dd93a5-2e67-4a6a-bb36-6bd944853f49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:23:37.015912 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dd93a5-2e67-4a6a-bb36-6bd944853f49 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:23:37.016195 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dd93a5-2e67-4a6a-bb36-6bd944853f49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:23:37.016473 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dd93a5-2e67-4a6a-bb36-6bd944853f49 admin admin] PATH_INFO: `/v3/domains` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:23:37.017232 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9dd93a5-2e67-4a6a-bb36-6bd944853f49 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 05:23:37.017586 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9dd93a5-2e67-4a6a-bb36-6bd944853f49 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:23:37.019207 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9dd93a5-2e67-4a6a-bb36-6bd944853f49 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:23:37.032473 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1237/2444] 10.222.0.23 () {58 vars in 1152 bytes} [Sat Jun 18 05:23:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 05:23:37.037031 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa284da-d246-4dad-be0c-d1da412dc0a3 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:23:37.046996 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa284da-d246-4dad-be0c-d1da412dc0a3 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:23:37.051749 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b7dd35-6d25-4287-99af-59546da97e50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:23:37.052403 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b7dd35-6d25-4287-99af-59546da97e50 admin admin] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:23:37.052689 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b7dd35-6d25-4287-99af-59546da97e50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:23:37.053530 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b7dd35-6d25-4287-99af-59546da97e50 admin admin] PATH_INFO: `/v3/projects` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:23:37.054283 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b7dd35-6d25-4287-99af-59546da97e50 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:23:37.056181 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b7dd35-6d25-4287-99af-59546da97e50 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:23:37.168165 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1208/2445] 10.222.0.23 () {60 vars in 1152 bytes} [Sat Jun 18 05:23:37 2022] POST /identity/v3/projects => generated 369 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:23:37.172477 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dd93a5-2e67-4a6a-bb36-6bd944853f49 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:23:37.183279 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dd93a5-2e67-4a6a-bb36-6bd944853f49 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:23:37.257510 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e15df90-7d6d-457e-9de2-4f432719c579 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:23:37.258346 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e15df90-7d6d-457e-9de2-4f432719c579 admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:23:37.258635 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e15df90-7d6d-457e-9de2-4f432719c579 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:23:37.258922 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e15df90-7d6d-457e-9de2-4f432719c579 admin admin] PATH_INFO: `/v3/users` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:23:37.259613 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e15df90-7d6d-457e-9de2-4f432719c579 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:23:37.261623 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e15df90-7d6d-457e-9de2-4f432719c579 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:23:37.350481 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1238/2446] 10.222.0.23 () {60 vars in 1143 bytes} [Sat Jun 18 05:23:37 2022] POST /identity/v3/users => generated 285 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 18 05:23:37.354701 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b7dd35-6d25-4287-99af-59546da97e50 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:23:37.368879 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b7dd35-6d25-4287-99af-59546da97e50 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:23:37.374551 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a15db42-0a5a-49ce-8161-3c3710d3ea70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:23:37.379617 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a15db42-0a5a-49ce-8161-3c3710d3ea70 admin admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:23:37.379617 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a15db42-0a5a-49ce-8161-3c3710d3ea70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:23:37.379617 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a15db42-0a5a-49ce-8161-3c3710d3ea70 admin admin] PATH_INFO: `/v3/roles` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:23:37.379617 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a15db42-0a5a-49ce-8161-3c3710d3ea70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:23:37.379617 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a15db42-0a5a-49ce-8161-3c3710d3ea70 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:23:37.390814 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1209/2447] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:23:37 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:23:37.394898 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e15df90-7d6d-457e-9de2-4f432719c579 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:23:37.405992 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e15df90-7d6d-457e-9de2-4f432719c579 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:23:37.411102 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c293cf-9944-4c83-8c41-397cbb4a0543 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:23:37.412646 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c293cf-9944-4c83-8c41-397cbb4a0543 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:23:37.412646 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c293cf-9944-4c83-8c41-397cbb4a0543 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:23:37.412883 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c293cf-9944-4c83-8c41-397cbb4a0543 admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:23:37.413784 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c293cf-9944-4c83-8c41-397cbb4a0543 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:23:37.415392 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c293cf-9944-4c83-8c41-397cbb4a0543 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:23:37.426748 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1239/2448] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:23:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:23:37.431694 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a15db42-0a5a-49ce-8161-3c3710d3ea70 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:23:37.442119 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a15db42-0a5a-49ce-8161-3c3710d3ea70 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:23:37.447920 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75cad61-e2c6-472f-82bb-9309e3f76d83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:23:37.448802 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75cad61-e2c6-472f-82bb-9309e3f76d83 admin admin] REQUEST_METHOD: `PUT` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:23:37.449205 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75cad61-e2c6-472f-82bb-9309e3f76d83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:23:37.449353 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75cad61-e2c6-472f-82bb-9309e3f76d83 admin admin] PATH_INFO: `/v3/projects/21c1946475f84235af1281d89ad00c8b/users/3361aef362a447d3ae5eb7f2d2e7ec31/roles/cbf5c4c26898413d93e73b46b41046a3` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:23:37.465075 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75cad61-e2c6-472f-82bb-9309e3f76d83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=21c1946475f84235af1281d89ad00c8b, user_id=3361aef362a447d3ae5eb7f2d2e7ec31, role_id=cbf5c4c26898413d93e73b46b41046a3)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:23:37.467822 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75cad61-e2c6-472f-82bb-9309e3f76d83 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:23:37.559764 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1210/2449] 10.222.0.23 () {60 vars in 1482 bytes} [Sat Jun 18 05:23:37 2022] PUT /identity/v3/projects/21c1946475f84235af1281d89ad00c8b/users/3361aef362a447d3ae5eb7f2d2e7ec31/roles/cbf5c4c26898413d93e73b46b41046a3 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:23:37.571952 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cb1cc5-3020-4237-a4ad-c3a6449653c7 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:23:37.572337 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cb1cc5-3020-4237-a4ad-c3a6449653c7 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:23:37.572627 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cb1cc5-3020-4237-a4ad-c3a6449653c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:23:37.606219 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-f5cb1cc5-3020-4237-a4ad-c3a6449653c7 None None] MFA Rules not processed for user `3361aef362a447d3ae5eb7f2d2e7ec31`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:23:37.606219 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-f5cb1cc5-3020-4237-a4ad-c3a6449653c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:23:37.636462 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:23:37.669905 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1240/2450] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:23:37 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:23:37.674483 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75cad61-e2c6-472f-82bb-9309e3f76d83 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:23:37.711913 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:23:37.741464 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75cad61-e2c6-472f-82bb-9309e3f76d83 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:23:37.746350 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398a3974-2674-4f73-9db3-35aed315e4dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:23:37.747009 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398a3974-2674-4f73-9db3-35aed315e4dd admin admin] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:23:37.747291 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398a3974-2674-4f73-9db3-35aed315e4dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:23:37.747564 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398a3974-2674-4f73-9db3-35aed315e4dd admin admin] PATH_INFO: `/v3/projects` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:23:37.748449 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398a3974-2674-4f73-9db3-35aed315e4dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:23:37.750432 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398a3974-2674-4f73-9db3-35aed315e4dd admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:23:37.846277 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1211/2451] 10.222.0.23 () {60 vars in 1152 bytes} [Sat Jun 18 05:23:37 2022] POST /identity/v3/projects => generated 367 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:23:37.851098 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cb1cc5-3020-4237-a4ad-c3a6449653c7 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:23:37.861922 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cb1cc5-3020-4237-a4ad-c3a6449653c7 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:23:37.917581 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b38cd7-1ee5-4cb5-ac2c-73a472641cb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:23:37.918297 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b38cd7-1ee5-4cb5-ac2c-73a472641cb6 admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:23:37.918531 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b38cd7-1ee5-4cb5-ac2c-73a472641cb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:23:37.918807 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b38cd7-1ee5-4cb5-ac2c-73a472641cb6 admin admin] PATH_INFO: `/v3/users` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:23:37.919522 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b38cd7-1ee5-4cb5-ac2c-73a472641cb6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:23:37.921513 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b38cd7-1ee5-4cb5-ac2c-73a472641cb6 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:23:38.026771 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1241/2452] 10.222.0.23 () {60 vars in 1143 bytes} [Sat Jun 18 05:23:37 2022] POST /identity/v3/users => generated 290 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:23:38.027855 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398a3974-2674-4f73-9db3-35aed315e4dd admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:23:38.041198 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398a3974-2674-4f73-9db3-35aed315e4dd admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:23:38.046029 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b824836-22f6-48e2-9781-1997887fd295 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:23:38.046684 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b824836-22f6-48e2-9781-1997887fd295 admin admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:23:38.046972 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b824836-22f6-48e2-9781-1997887fd295 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:23:38.047254 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b824836-22f6-48e2-9781-1997887fd295 admin admin] PATH_INFO: `/v3/roles` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:23:38.047985 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b824836-22f6-48e2-9781-1997887fd295 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:23:38.049663 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b824836-22f6-48e2-9781-1997887fd295 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:23:38.061528 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1212/2453] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:23:38 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:23:38.065689 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b38cd7-1ee5-4cb5-ac2c-73a472641cb6 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:23:38.076183 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b38cd7-1ee5-4cb5-ac2c-73a472641cb6 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:23:38.080865 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b544ee96-2d08-4331-b1ac-50c79102fa59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:23:38.081771 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b544ee96-2d08-4331-b1ac-50c79102fa59 admin admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:23:38.082063 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b544ee96-2d08-4331-b1ac-50c79102fa59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:23:38.082342 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b544ee96-2d08-4331-b1ac-50c79102fa59 admin admin] PATH_INFO: `/v3/projects/7a5cb5661c624dcb9e3db0f7572a3f6c/users/691d1cc44e0d40e9940a7edff8b7f734/roles/21c65f2c51bd4dd283d13507d2714467` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:23:38.099013 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b544ee96-2d08-4331-b1ac-50c79102fa59 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a5cb5661c624dcb9e3db0f7572a3f6c, user_id=691d1cc44e0d40e9940a7edff8b7f734, role_id=21c65f2c51bd4dd283d13507d2714467)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:23:38.102979 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b544ee96-2d08-4331-b1ac-50c79102fa59 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:23:38.213455 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1242/2454] 10.222.0.23 () {60 vars in 1482 bytes} [Sat Jun 18 05:23:38 2022] PUT /identity/v3/projects/7a5cb5661c624dcb9e3db0f7572a3f6c/users/691d1cc44e0d40e9940a7edff8b7f734/roles/21c65f2c51bd4dd283d13507d2714467 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:23:38.217470 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b824836-22f6-48e2-9781-1997887fd295 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:23:38.227708 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b824836-22f6-48e2-9781-1997887fd295 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:23:38.284324 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0002ae9-02a5-4949-8e9d-9b7a215284fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:23:38.284972 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0002ae9-02a5-4949-8e9d-9b7a215284fe admin admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:23:38.285282 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0002ae9-02a5-4949-8e9d-9b7a215284fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:23:38.285584 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0002ae9-02a5-4949-8e9d-9b7a215284fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:23:38.286295 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0002ae9-02a5-4949-8e9d-9b7a215284fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:23:38.288401 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0002ae9-02a5-4949-8e9d-9b7a215284fe admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:23:38.297900 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1213/2455] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:23:38 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:23:38.302263 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b544ee96-2d08-4331-b1ac-50c79102fa59 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:23:38.312638 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b544ee96-2d08-4331-b1ac-50c79102fa59 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:23:38.317301 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea1eabe-f4df-4aa1-86b5-d251aac0039a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:23:38.318203 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea1eabe-f4df-4aa1-86b5-d251aac0039a admin admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:23:38.318977 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea1eabe-f4df-4aa1-86b5-d251aac0039a admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:23:38.319275 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea1eabe-f4df-4aa1-86b5-d251aac0039a admin admin] PATH_INFO: `/v3/projects/7a5cb5661c624dcb9e3db0f7572a3f6c/users/691d1cc44e0d40e9940a7edff8b7f734/roles/21c65f2c51bd4dd283d13507d2714467` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:23:38.321261 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea1eabe-f4df-4aa1-86b5-d251aac0039a admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a5cb5661c624dcb9e3db0f7572a3f6c, user_id=691d1cc44e0d40e9940a7edff8b7f734, role_id=21c65f2c51bd4dd283d13507d2714467)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:23:38.323925 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea1eabe-f4df-4aa1-86b5-d251aac0039a admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:23:38.334995 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1243/2456] 10.222.0.23 () {60 vars in 1482 bytes} [Sat Jun 18 05:23:38 2022] PUT /identity/v3/projects/7a5cb5661c624dcb9e3db0f7572a3f6c/users/691d1cc44e0d40e9940a7edff8b7f734/roles/21c65f2c51bd4dd283d13507d2714467 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:23:38.345026 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8ffdec-a1ec-48b5-a51e-a15c4138a3a9 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:23:38.345322 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8ffdec-a1ec-48b5-a51e-a15c4138a3a9 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:23:38.345635 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8ffdec-a1ec-48b5-a51e-a15c4138a3a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:23:38.371508 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-4f8ffdec-a1ec-48b5-a51e-a15c4138a3a9 None None] MFA Rules not processed for user `691d1cc44e0d40e9940a7edff8b7f734`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:23:38.372719 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-4f8ffdec-a1ec-48b5-a51e-a15c4138a3a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:23:38.441472 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1214/2457] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:23:38 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:23:38.452873 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea1eabe-f4df-4aa1-86b5-d251aac0039a admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:23:38.454625 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-eea1eabe-f4df-4aa1-86b5-d251aac0039a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:23:38.515593 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea1eabe-f4df-4aa1-86b5-d251aac0039a admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:23:38.519240 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906322b8-432d-4b93-be67-5cf4c71f129e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9de0428abc8400aa338aba4f7c34c94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:23:38.519888 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906322b8-432d-4b93-be67-5cf4c71f129e service cinder] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:23:38.520172 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906322b8-432d-4b93-be67-5cf4c71f129e service cinder] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:23:38.520448 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906322b8-432d-4b93-be67-5cf4c71f129e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:23:38.529631 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906322b8-432d-4b93-be67-5cf4c71f129e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:23:38.531359 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906322b8-432d-4b93-be67-5cf4c71f129e service cinder] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:23:38.540943 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1244/2458] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:23:38 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:23:41.274188 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8ffdec-a1ec-48b5-a51e-a15c4138a3a9 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:23:41.275732 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-4f8ffdec-a1ec-48b5-a51e-a15c4138a3a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:23:41.320968 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:23:41.363647 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8ffdec-a1ec-48b5-a51e-a15c4138a3a9 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:23:41.369358 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfe63d9-dfcd-46f5-9a9c-ff8da5663539 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c4d7cde8f4ad9aa479bb09e325db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:23:41.370404 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfe63d9-dfcd-46f5-9a9c-ff8da5663539 service neutron] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:23:41.370917 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfe63d9-dfcd-46f5-9a9c-ff8da5663539 service neutron] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:23:41.371439 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfe63d9-dfcd-46f5-9a9c-ff8da5663539 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:23:41.380375 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-1cfe63d9-dfcd-46f5-9a9c-ff8da5663539 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:23:41.467590 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfe63d9-dfcd-46f5-9a9c-ff8da5663539 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:23:41.468970 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfe63d9-dfcd-46f5-9a9c-ff8da5663539 service neutron] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:23:41.479726 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1215/2459] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:23:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:23:41.681883 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906322b8-432d-4b93-be67-5cf4c71f129e service cinder] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:23:41.692829 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906322b8-432d-4b93-be67-5cf4c71f129e service cinder] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:23:41.698259 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2563211-a032-40a1-a840-8c7803d664c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:23:41.699238 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2563211-a032-40a1-a840-8c7803d664c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:23:41.699595 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2563211-a032-40a1-a840-8c7803d664c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:23:41.699940 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2563211-a032-40a1-a840-8c7803d664c9 admin admin] PATH_INFO: `/v3/users/3361aef362a447d3ae5eb7f2d2e7ec31` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:23:41.701247 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2563211-a032-40a1-a840-8c7803d664c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3361aef362a447d3ae5eb7f2d2e7ec31)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:23:41.703422 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2563211-a032-40a1-a840-8c7803d664c9 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:23:41.729229 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=91,keystone:INSERT=5,keystone:DELETE=4 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:23:41.843849 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-b2563211-a032-40a1-a840-8c7803d664c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3361aef362a447d3ae5eb7f2d2e7ec31'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:23:41.856952 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-b2563211-a032-40a1-a840-8c7803d664c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3361aef362a447d3ae5eb7f2d2e7ec31'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:23:41.857707 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-b2563211-a032-40a1-a840-8c7803d664c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3361aef362a447d3ae5eb7f2d2e7ec31'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:23:41.909303 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-b2563211-a032-40a1-a840-8c7803d664c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3361aef362a447d3ae5eb7f2d2e7ec31'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:23:41.917516 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-b2563211-a032-40a1-a840-8c7803d664c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3361aef362a447d3ae5eb7f2d2e7ec31'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:23:41.924142 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1245/2460] 10.222.0.23 () {58 vars in 1223 bytes} [Sat Jun 18 05:23:41 2022] DELETE /identity/v3/users/3361aef362a447d3ae5eb7f2d2e7ec31 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 18 05:23:41.929970 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a30771b-382c-4029-81ad-0302603bad86 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:23:41.930338 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a30771b-382c-4029-81ad-0302603bad86 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:23:41.930706 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a30771b-382c-4029-81ad-0302603bad86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:23:41.948376 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-0a30771b-382c-4029-81ad-0302603bad86 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:23:41.949610 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-0a30771b-382c-4029-81ad-0302603bad86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:23:42.013315 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1216/2461] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:23:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:23:42.023357 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2563211-a032-40a1-a840-8c7803d664c9 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:23:42.025565 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-b2563211-a032-40a1-a840-8c7803d664c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:23:42.087738 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2563211-a032-40a1-a840-8c7803d664c9 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:23:42.091450 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cf5ea1-d54b-4879-9c16-73e951b0b684 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c4d7cde8f4ad9aa479bb09e325db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:23:42.092056 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cf5ea1-d54b-4879-9c16-73e951b0b684 service neutron] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:23:42.092339 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cf5ea1-d54b-4879-9c16-73e951b0b684 service neutron] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:23:42.092618 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cf5ea1-d54b-4879-9c16-73e951b0b684 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:23:42.102941 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cf5ea1-d54b-4879-9c16-73e951b0b684 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:23:42.104501 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cf5ea1-d54b-4879-9c16-73e951b0b684 service neutron] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:23:42.115059 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1246/2462] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:23:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 18 05:23:42.519999 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a30771b-382c-4029-81ad-0302603bad86 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:23:42.521987 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-0a30771b-382c-4029-81ad-0302603bad86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:23:42.536399 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a30771b-382c-4029-81ad-0302603bad86 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:23:42.554534 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638fef57-d06f-485f-b872-942482091d49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:23:42.555441 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638fef57-d06f-485f-b872-942482091d49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:23:42.555732 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638fef57-d06f-485f-b872-942482091d49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:23:42.556015 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638fef57-d06f-485f-b872-942482091d49 admin admin] PATH_INFO: `/v3/projects/21c1946475f84235af1281d89ad00c8b` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:23:42.557667 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638fef57-d06f-485f-b872-942482091d49 admin admin] RBAC: Authorizing `identity:delete_project(project_id=21c1946475f84235af1281d89ad00c8b)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:23:42.559724 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638fef57-d06f-485f-b872-942482091d49 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:23:42.664874 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-638fef57-d06f-485f-b872-942482091d49 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '21c1946475f84235af1281d89ad00c8b', 'request_id': 'req-638fef57-d06f-485f-b872-942482091d49'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:23:42.670488 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-638fef57-d06f-485f-b872-942482091d49 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '21c1946475f84235af1281d89ad00c8b', 'request_id': 'req-638fef57-d06f-485f-b872-942482091d49'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:23:42.743189 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-638fef57-d06f-485f-b872-942482091d49 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '21c1946475f84235af1281d89ad00c8b', 'request_id': 'req-638fef57-d06f-485f-b872-942482091d49'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:23:42.751726 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:23:42.755641 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-638fef57-d06f-485f-b872-942482091d49 admin admin] The token cache is being invalidate because project 21c1946475f84235af1281d89ad00c8b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80468) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 18 05:23:42.756477 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-638fef57-d06f-485f-b872-942482091d49 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:23:42.757241 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-638fef57-d06f-485f-b872-942482091d49 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:23:42.758824 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1217/2463] 10.222.0.23 () {58 vars in 1232 bytes} [Sat Jun 18 05:23:42 2022] DELETE /identity/v3/projects/21c1946475f84235af1281d89ad00c8b => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:23:42.763602 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cf5ea1-d54b-4879-9c16-73e951b0b684 service neutron] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:23:42.773928 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cf5ea1-d54b-4879-9c16-73e951b0b684 service neutron] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:23:42.833980 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a73094-6f8e-4f38-b0eb-77f5eaba7d7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:23:42.835478 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a73094-6f8e-4f38-b0eb-77f5eaba7d7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:23:42.835831 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a73094-6f8e-4f38-b0eb-77f5eaba7d7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:23:42.836183 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a73094-6f8e-4f38-b0eb-77f5eaba7d7e admin admin] PATH_INFO: `/v3/users/691d1cc44e0d40e9940a7edff8b7f734` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:23:42.837507 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a73094-6f8e-4f38-b0eb-77f5eaba7d7e admin admin] RBAC: Authorizing `identity:delete_user(user_id=691d1cc44e0d40e9940a7edff8b7f734)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:23:42.839478 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a73094-6f8e-4f38-b0eb-77f5eaba7d7e admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:23:43.026298 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:23:43.028337 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-f8a73094-6f8e-4f38-b0eb-77f5eaba7d7e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '691d1cc44e0d40e9940a7edff8b7f734'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:23:43.041275 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-f8a73094-6f8e-4f38-b0eb-77f5eaba7d7e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '691d1cc44e0d40e9940a7edff8b7f734'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:23:43.042053 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-f8a73094-6f8e-4f38-b0eb-77f5eaba7d7e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '691d1cc44e0d40e9940a7edff8b7f734'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:23:43.113236 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-f8a73094-6f8e-4f38-b0eb-77f5eaba7d7e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '691d1cc44e0d40e9940a7edff8b7f734'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:23:43.121843 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-f8a73094-6f8e-4f38-b0eb-77f5eaba7d7e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '691d1cc44e0d40e9940a7edff8b7f734'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:23:43.132103 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1247/2464] 10.222.0.23 () {58 vars in 1223 bytes} [Sat Jun 18 05:23:42 2022] DELETE /identity/v3/users/691d1cc44e0d40e9940a7edff8b7f734 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:23:43.333555 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638fef57-d06f-485f-b872-942482091d49 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:23:43.339034 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-638fef57-d06f-485f-b872-942482091d49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:23:43.495476 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638fef57-d06f-485f-b872-942482091d49 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:23:43.589881 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2924f791-0917-4c0d-8e63-46a6f1b4f1c1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15e7f45ee0744c1d9602187eac8e8b80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:23:43.590914 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2924f791-0917-4c0d-8e63-46a6f1b4f1c1 service nova] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:23:43.591205 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2924f791-0917-4c0d-8e63-46a6f1b4f1c1 service nova] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:23:43.591482 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2924f791-0917-4c0d-8e63-46a6f1b4f1c1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:23:43.593342 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-2924f791-0917-4c0d-8e63-46a6f1b4f1c1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:23:43.606478 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2924f791-0917-4c0d-8e63-46a6f1b4f1c1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:23:43.607953 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2924f791-0917-4c0d-8e63-46a6f1b4f1c1 service nova] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:23:43.621786 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1218/2465] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:23:43 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:23:43.878808 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a73094-6f8e-4f38-b0eb-77f5eaba7d7e admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:23:43.880674 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-f8a73094-6f8e-4f38-b0eb-77f5eaba7d7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:23:43.960564 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a73094-6f8e-4f38-b0eb-77f5eaba7d7e admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:23:43.968662 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a82710a-12bc-4533-9c94-21f788333c3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:23:43.969570 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a82710a-12bc-4533-9c94-21f788333c3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:23:43.969855 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a82710a-12bc-4533-9c94-21f788333c3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:23:43.970130 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a82710a-12bc-4533-9c94-21f788333c3a admin admin] PATH_INFO: `/v3/projects/7a5cb5661c624dcb9e3db0f7572a3f6c` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:23:43.971347 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a82710a-12bc-4533-9c94-21f788333c3a admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a5cb5661c624dcb9e3db0f7572a3f6c)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:23:43.973411 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a82710a-12bc-4533-9c94-21f788333c3a admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:23:44.079296 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-5a82710a-12bc-4533-9c94-21f788333c3a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a5cb5661c624dcb9e3db0f7572a3f6c', 'request_id': 'req-5a82710a-12bc-4533-9c94-21f788333c3a'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:23:44.096606 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-5a82710a-12bc-4533-9c94-21f788333c3a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a5cb5661c624dcb9e3db0f7572a3f6c', 'request_id': 'req-5a82710a-12bc-4533-9c94-21f788333c3a'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:23:44.107916 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-5a82710a-12bc-4533-9c94-21f788333c3a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a5cb5661c624dcb9e3db0f7572a3f6c', 'request_id': 'req-5a82710a-12bc-4533-9c94-21f788333c3a'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:23:44.213309 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-5a82710a-12bc-4533-9c94-21f788333c3a admin admin] The token cache is being invalidate because project 7a5cb5661c624dcb9e3db0f7572a3f6c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80469) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 18 05:23:44.213309 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-5a82710a-12bc-4533-9c94-21f788333c3a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:23:44.213556 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-5a82710a-12bc-4533-9c94-21f788333c3a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:23:44.221834 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1248/2466] 10.222.0.23 () {58 vars in 1232 bytes} [Sat Jun 18 05:23:43 2022] DELETE /identity/v3/projects/7a5cb5661c624dcb9e3db0f7572a3f6c => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:23:44.223656 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd08d6b-899b-4494-a342-8b890ea4c4ab None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:23:44.223952 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd08d6b-899b-4494-a342-8b890ea4c4ab None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:23:44.224223 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd08d6b-899b-4494-a342-8b890ea4c4ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:23:44.260666 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-3fd08d6b-899b-4494-a342-8b890ea4c4ab None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:23:44.262994 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-3fd08d6b-899b-4494-a342-8b890ea4c4ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:23:44.442466 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1219/2467] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:23:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:23:44.455604 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa214045-9c64-446d-ae66-b73b2cce8393 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:23:44.456754 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa214045-9c64-446d-ae66-b73b2cce8393 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:23:44.457080 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa214045-9c64-446d-ae66-b73b2cce8393 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:23:44.495402 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-aa214045-9c64-446d-ae66-b73b2cce8393 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:23:44.497979 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-aa214045-9c64-446d-ae66-b73b2cce8393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:23:44.516932 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1249/2468] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:23:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:23:44.525455 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd08d6b-899b-4494-a342-8b890ea4c4ab None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:23:44.536054 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd08d6b-899b-4494-a342-8b890ea4c4ab None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:23:44.542406 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028a8a41-524c-4aef-b0e1-90515f9de6c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:23:44.545171 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028a8a41-524c-4aef-b0e1-90515f9de6c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:23:44.545729 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028a8a41-524c-4aef-b0e1-90515f9de6c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:23:44.546312 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028a8a41-524c-4aef-b0e1-90515f9de6c1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:23:44.547260 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028a8a41-524c-4aef-b0e1-90515f9de6c1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 05:23:44.547778 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028a8a41-524c-4aef-b0e1-90515f9de6c1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:23:44.549903 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028a8a41-524c-4aef-b0e1-90515f9de6c1 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:23:44.567250 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1220/2469] 10.222.0.23 () {58 vars in 1152 bytes} [Sat Jun 18 05:23:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 05:23:44.571854 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa214045-9c64-446d-ae66-b73b2cce8393 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:23:44.583280 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa214045-9c64-446d-ae66-b73b2cce8393 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:23:44.590978 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4362be-02e5-4189-98a0-722b8898165f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:23:44.591942 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4362be-02e5-4189-98a0-722b8898165f admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:23:44.596924 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4362be-02e5-4189-98a0-722b8898165f admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:23:44.596924 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4362be-02e5-4189-98a0-722b8898165f admin admin] PATH_INFO: `/v3/projects` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:23:44.596924 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4362be-02e5-4189-98a0-722b8898165f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:23:44.596924 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4362be-02e5-4189-98a0-722b8898165f admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:23:44.779102 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1250/2470] 10.222.0.23 () {60 vars in 1152 bytes} [Sat Jun 18 05:23:44 2022] POST /identity/v3/projects => generated 369 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:23:44.784283 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028a8a41-524c-4aef-b0e1-90515f9de6c1 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:23:44.799289 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028a8a41-524c-4aef-b0e1-90515f9de6c1 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:23:44.811178 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:23:44.884519 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedf2e22-8596-42cd-abee-15d838fe4510 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:23:44.886064 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedf2e22-8596-42cd-abee-15d838fe4510 admin admin] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:23:44.890342 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedf2e22-8596-42cd-abee-15d838fe4510 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:23:44.890342 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedf2e22-8596-42cd-abee-15d838fe4510 admin admin] PATH_INFO: `/v3/users` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:23:44.890342 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedf2e22-8596-42cd-abee-15d838fe4510 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:23:44.890342 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedf2e22-8596-42cd-abee-15d838fe4510 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:23:44.990015 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1221/2471] 10.222.0.23 () {60 vars in 1143 bytes} [Sat Jun 18 05:23:44 2022] POST /identity/v3/users => generated 285 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:23:44.995664 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4362be-02e5-4189-98a0-722b8898165f admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:23:45.010985 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4362be-02e5-4189-98a0-722b8898165f admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:23:45.016835 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7947d23-f0c4-4265-aa38-65e695c1c1a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:23:45.017537 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7947d23-f0c4-4265-aa38-65e695c1c1a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:23:45.017823 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7947d23-f0c4-4265-aa38-65e695c1c1a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:23:45.018123 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7947d23-f0c4-4265-aa38-65e695c1c1a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:23:45.018857 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7947d23-f0c4-4265-aa38-65e695c1c1a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:23:45.020447 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7947d23-f0c4-4265-aa38-65e695c1c1a5 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:23:45.032257 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1251/2472] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:23:44 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:23:45.036221 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedf2e22-8596-42cd-abee-15d838fe4510 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:23:45.049105 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedf2e22-8596-42cd-abee-15d838fe4510 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:23:45.054798 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddc38ca-a28c-4685-b59e-20238bf555d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:23:45.055466 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddc38ca-a28c-4685-b59e-20238bf555d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:23:45.055820 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddc38ca-a28c-4685-b59e-20238bf555d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:23:45.056099 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddc38ca-a28c-4685-b59e-20238bf555d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:23:45.056838 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddc38ca-a28c-4685-b59e-20238bf555d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:23:45.058537 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddc38ca-a28c-4685-b59e-20238bf555d4 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:23:45.069309 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1222/2473] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:23:45 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:23:45.073507 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7947d23-f0c4-4265-aa38-65e695c1c1a5 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:23:45.083853 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7947d23-f0c4-4265-aa38-65e695c1c1a5 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:23:45.089768 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353e18e2-3e95-4bed-a294-f7a9a5107a88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:23:45.089768 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353e18e2-3e95-4bed-a294-f7a9a5107a88 admin admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:23:45.089768 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353e18e2-3e95-4bed-a294-f7a9a5107a88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:23:45.090090 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353e18e2-3e95-4bed-a294-f7a9a5107a88 admin admin] PATH_INFO: `/v3/projects/b2d41818afeb408fbccabda446fb4ad5/users/826caae04ee4421abe88c7539050c0c9/roles/cbf5c4c26898413d93e73b46b41046a3` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:23:45.102597 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:23:45.106972 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353e18e2-3e95-4bed-a294-f7a9a5107a88 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2d41818afeb408fbccabda446fb4ad5, user_id=826caae04ee4421abe88c7539050c0c9, role_id=cbf5c4c26898413d93e73b46b41046a3)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:23:45.109682 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353e18e2-3e95-4bed-a294-f7a9a5107a88 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:23:45.183056 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1252/2474] 10.222.0.23 () {60 vars in 1482 bytes} [Sat Jun 18 05:23:45 2022] PUT /identity/v3/projects/b2d41818afeb408fbccabda446fb4ad5/users/826caae04ee4421abe88c7539050c0c9/roles/cbf5c4c26898413d93e73b46b41046a3 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:23:45.193633 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2fb5ea-fe18-4c99-897f-bfac18517501 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:23:45.194326 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2fb5ea-fe18-4c99-897f-bfac18517501 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:23:45.194447 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2fb5ea-fe18-4c99-897f-bfac18517501 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:23:45.223501 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-cf2fb5ea-fe18-4c99-897f-bfac18517501 None None] MFA Rules not processed for user `826caae04ee4421abe88c7539050c0c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:23:45.225304 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-cf2fb5ea-fe18-4c99-897f-bfac18517501 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:23:45.293370 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1223/2475] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:23:45 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:23:45.297839 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353e18e2-3e95-4bed-a294-f7a9a5107a88 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:23:45.390675 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353e18e2-3e95-4bed-a294-f7a9a5107a88 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:23:45.396904 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fe0163-18f2-46ac-a7e8-f7d66ccf99d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:23:45.397668 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fe0163-18f2-46ac-a7e8-f7d66ccf99d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:23:45.398081 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fe0163-18f2-46ac-a7e8-f7d66ccf99d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:23:45.401188 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fe0163-18f2-46ac-a7e8-f7d66ccf99d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:23:45.401188 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fe0163-18f2-46ac-a7e8-f7d66ccf99d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:23:45.402087 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fe0163-18f2-46ac-a7e8-f7d66ccf99d1 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:23:45.532551 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1253/2476] 10.222.0.23 () {60 vars in 1152 bytes} [Sat Jun 18 05:23:45 2022] POST /identity/v3/projects => generated 369 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:23:45.536870 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2fb5ea-fe18-4c99-897f-bfac18517501 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:23:45.550595 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2fb5ea-fe18-4c99-897f-bfac18517501 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:23:45.624615 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ae63d9-244b-4591-a9ec-b3416685e109 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:23:45.625354 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ae63d9-244b-4591-a9ec-b3416685e109 admin admin] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:23:45.625739 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ae63d9-244b-4591-a9ec-b3416685e109 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:23:45.626086 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ae63d9-244b-4591-a9ec-b3416685e109 admin admin] PATH_INFO: `/v3/users` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:23:45.627217 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ae63d9-244b-4591-a9ec-b3416685e109 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:23:45.630369 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ae63d9-244b-4591-a9ec-b3416685e109 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:23:45.719143 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1224/2477] 10.222.0.23 () {60 vars in 1143 bytes} [Sat Jun 18 05:23:45 2022] POST /identity/v3/users => generated 291 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:23:45.723112 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fe0163-18f2-46ac-a7e8-f7d66ccf99d1 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:23:45.733878 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fe0163-18f2-46ac-a7e8-f7d66ccf99d1 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:23:45.739587 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e817fa-56e2-48cb-a7f5-575ccabd317c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:23:45.740834 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e817fa-56e2-48cb-a7f5-575ccabd317c admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:23:45.741115 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e817fa-56e2-48cb-a7f5-575ccabd317c admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:23:45.741415 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e817fa-56e2-48cb-a7f5-575ccabd317c admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:23:45.745059 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e817fa-56e2-48cb-a7f5-575ccabd317c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:23:45.745059 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e817fa-56e2-48cb-a7f5-575ccabd317c admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:23:45.755308 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1254/2478] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:23:45 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:23:45.759383 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ae63d9-244b-4591-a9ec-b3416685e109 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:23:45.769594 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ae63d9-244b-4591-a9ec-b3416685e109 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:23:45.774514 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9a91de-01f7-459e-9dc9-c2c92aaf52f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:23:45.775379 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9a91de-01f7-459e-9dc9-c2c92aaf52f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:23:45.775733 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9a91de-01f7-459e-9dc9-c2c92aaf52f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:23:45.776131 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9a91de-01f7-459e-9dc9-c2c92aaf52f4 admin admin] PATH_INFO: `/v3/projects/edf89c88b9864058882d81837bae11f9/users/1964ca0181384d35a53e9bf7505ecba6/roles/21c65f2c51bd4dd283d13507d2714467` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:23:45.779212 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:23:45.791616 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9a91de-01f7-459e-9dc9-c2c92aaf52f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=edf89c88b9864058882d81837bae11f9, user_id=1964ca0181384d35a53e9bf7505ecba6, role_id=21c65f2c51bd4dd283d13507d2714467)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:23:45.794335 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9a91de-01f7-459e-9dc9-c2c92aaf52f4 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:23:45.875229 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1225/2479] 10.222.0.23 () {60 vars in 1482 bytes} [Sat Jun 18 05:23:45 2022] PUT /identity/v3/projects/edf89c88b9864058882d81837bae11f9/users/1964ca0181384d35a53e9bf7505ecba6/roles/21c65f2c51bd4dd283d13507d2714467 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:23:45.879278 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e817fa-56e2-48cb-a7f5-575ccabd317c admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:23:45.889809 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e817fa-56e2-48cb-a7f5-575ccabd317c admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:23:45.956790 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f71e38-ed44-4647-8dd7-0628baab33d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:23:45.958294 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f71e38-ed44-4647-8dd7-0628baab33d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:23:45.958294 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f71e38-ed44-4647-8dd7-0628baab33d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:23:45.958294 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f71e38-ed44-4647-8dd7-0628baab33d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:23:45.963055 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f71e38-ed44-4647-8dd7-0628baab33d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:23:45.963055 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f71e38-ed44-4647-8dd7-0628baab33d8 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:23:45.976550 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1255/2480] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:23:45 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:23:45.977999 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9a91de-01f7-459e-9dc9-c2c92aaf52f4 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:23:45.989003 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9a91de-01f7-459e-9dc9-c2c92aaf52f4 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:23:45.994856 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a82e16a-1017-4ad5-b51e-da940603dc6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:23:45.996353 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a82e16a-1017-4ad5-b51e-da940603dc6c admin admin] REQUEST_METHOD: `PUT` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:23:45.996725 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a82e16a-1017-4ad5-b51e-da940603dc6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:23:45.997079 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a82e16a-1017-4ad5-b51e-da940603dc6c admin admin] PATH_INFO: `/v3/projects/edf89c88b9864058882d81837bae11f9/users/1964ca0181384d35a53e9bf7505ecba6/roles/21c65f2c51bd4dd283d13507d2714467` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:23:46.002350 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a82e16a-1017-4ad5-b51e-da940603dc6c admin admin] RBAC: Authorizing `identity:create_grant(project_id=edf89c88b9864058882d81837bae11f9, user_id=1964ca0181384d35a53e9bf7505ecba6, role_id=21c65f2c51bd4dd283d13507d2714467)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:23:46.005183 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a82e16a-1017-4ad5-b51e-da940603dc6c admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:23:46.016964 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1226/2481] 10.222.0.23 () {60 vars in 1482 bytes} [Sat Jun 18 05:23:45 2022] PUT /identity/v3/projects/edf89c88b9864058882d81837bae11f9/users/1964ca0181384d35a53e9bf7505ecba6/roles/21c65f2c51bd4dd283d13507d2714467 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:23:46.029199 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b763fdb1-6cef-40a6-810d-b2a994ab9122 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:23:46.030590 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b763fdb1-6cef-40a6-810d-b2a994ab9122 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:23:46.030988 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b763fdb1-6cef-40a6-810d-b2a994ab9122 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:23:46.066173 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-b763fdb1-6cef-40a6-810d-b2a994ab9122 None None] MFA Rules not processed for user `1964ca0181384d35a53e9bf7505ecba6`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:23:46.066173 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-b763fdb1-6cef-40a6-810d-b2a994ab9122 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:23:46.128982 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:23:46.151555 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1256/2482] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:23:46 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:23:46.165287 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a82e16a-1017-4ad5-b51e-da940603dc6c admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:23:46.166987 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-4a82e16a-1017-4ad5-b51e-da940603dc6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:23:46.262227 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a82e16a-1017-4ad5-b51e-da940603dc6c admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:23:46.269218 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a2d10d-008e-4b3e-bcba-571a2c66e2c1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9de0428abc8400aa338aba4f7c34c94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:23:46.270147 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a2d10d-008e-4b3e-bcba-571a2c66e2c1 service cinder] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:23:46.270441 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a2d10d-008e-4b3e-bcba-571a2c66e2c1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:23:46.270720 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a2d10d-008e-4b3e-bcba-571a2c66e2c1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:23:46.288539 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a2d10d-008e-4b3e-bcba-571a2c66e2c1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:23:46.293590 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a2d10d-008e-4b3e-bcba-571a2c66e2c1 service cinder] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:23:46.333592 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1227/2483] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:23:46 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:23:46.437528 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b763fdb1-6cef-40a6-810d-b2a994ab9122 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:23:46.439071 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-b763fdb1-6cef-40a6-810d-b2a994ab9122 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:23:46.452538 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b763fdb1-6cef-40a6-810d-b2a994ab9122 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:23:46.456269 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a49e34-5950-4aea-8c92-54c78c745e8d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9de0428abc8400aa338aba4f7c34c94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:23:46.456989 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a49e34-5950-4aea-8c92-54c78c745e8d service cinder] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:23:46.457268 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a49e34-5950-4aea-8c92-54c78c745e8d service cinder] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:23:46.457624 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a49e34-5950-4aea-8c92-54c78c745e8d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:23:46.517859 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a49e34-5950-4aea-8c92-54c78c745e8d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:23:46.519645 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a49e34-5950-4aea-8c92-54c78c745e8d service cinder] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:23:46.530178 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1257/2484] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:23:46 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:23:46.544134 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a2d10d-008e-4b3e-bcba-571a2c66e2c1 service cinder] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:23:46.545957 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-e6a2d10d-008e-4b3e-bcba-571a2c66e2c1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:23:46.635830 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a2d10d-008e-4b3e-bcba-571a2c66e2c1 service cinder] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:23:46.642455 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d93dfd-274d-4ab4-b59d-9d27d4002280 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:23:46.643325 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d93dfd-274d-4ab4-b59d-9d27d4002280 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:23:46.643606 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d93dfd-274d-4ab4-b59d-9d27d4002280 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:23:46.643882 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d93dfd-274d-4ab4-b59d-9d27d4002280 admin admin] PATH_INFO: `/v3/users/d2c8f964ee6a49cda501ed28a9432c8e` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:23:46.645160 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d93dfd-274d-4ab4-b59d-9d27d4002280 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d2c8f964ee6a49cda501ed28a9432c8e)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:23:46.647569 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d93dfd-274d-4ab4-b59d-9d27d4002280 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:23:46.861090 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-32d93dfd-274d-4ab4-b59d-9d27d4002280 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd2c8f964ee6a49cda501ed28a9432c8e'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:23:46.861867 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-32d93dfd-274d-4ab4-b59d-9d27d4002280 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd2c8f964ee6a49cda501ed28a9432c8e'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:23:46.949092 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-32d93dfd-274d-4ab4-b59d-9d27d4002280 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd2c8f964ee6a49cda501ed28a9432c8e'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:23:46.962438 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-32d93dfd-274d-4ab4-b59d-9d27d4002280 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd2c8f964ee6a49cda501ed28a9432c8e'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:23:46.967823 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:23:46.971202 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-32d93dfd-274d-4ab4-b59d-9d27d4002280 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd2c8f964ee6a49cda501ed28a9432c8e'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:23:46.977694 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1228/2485] 10.222.0.23 () {58 vars in 1223 bytes} [Sat Jun 18 05:23:46 2022] DELETE /identity/v3/users/d2c8f964ee6a49cda501ed28a9432c8e => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:23:47.378074 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a49e34-5950-4aea-8c92-54c78c745e8d service cinder] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:23:47.395749 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a49e34-5950-4aea-8c92-54c78c745e8d service cinder] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:23:47.477511 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddde3819-4025-455a-b7b3-d396e765bfdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:23:47.481007 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddde3819-4025-455a-b7b3-d396e765bfdb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:23:47.481007 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddde3819-4025-455a-b7b3-d396e765bfdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:23:47.481007 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddde3819-4025-455a-b7b3-d396e765bfdb admin admin] PATH_INFO: `/v3/projects/969c470f720c4b2f97ba9f45c5d0d426` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:23:47.481327 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddde3819-4025-455a-b7b3-d396e765bfdb admin admin] RBAC: Authorizing `identity:delete_project(project_id=969c470f720c4b2f97ba9f45c5d0d426)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:23:47.483233 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddde3819-4025-455a-b7b3-d396e765bfdb admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:23:47.617216 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-ddde3819-4025-455a-b7b3-d396e765bfdb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '969c470f720c4b2f97ba9f45c5d0d426', 'request_id': 'req-ddde3819-4025-455a-b7b3-d396e765bfdb'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:23:47.623134 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-ddde3819-4025-455a-b7b3-d396e765bfdb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '969c470f720c4b2f97ba9f45c5d0d426', 'request_id': 'req-ddde3819-4025-455a-b7b3-d396e765bfdb'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:23:47.630090 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-ddde3819-4025-455a-b7b3-d396e765bfdb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '969c470f720c4b2f97ba9f45c5d0d426', 'request_id': 'req-ddde3819-4025-455a-b7b3-d396e765bfdb'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:23:47.735681 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-ddde3819-4025-455a-b7b3-d396e765bfdb admin admin] The token cache is being invalidate because project 969c470f720c4b2f97ba9f45c5d0d426 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80469) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 18 05:23:47.735969 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-ddde3819-4025-455a-b7b3-d396e765bfdb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:23:47.739679 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-ddde3819-4025-455a-b7b3-d396e765bfdb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:23:47.739679 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1258/2486] 10.222.0.23 () {58 vars in 1232 bytes} [Sat Jun 18 05:23:47 2022] DELETE /identity/v3/projects/969c470f720c4b2f97ba9f45c5d0d426 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:23:47.744156 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2782eb3-a24c-4cb4-a109-05b540f6d706 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:23:47.744447 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2782eb3-a24c-4cb4-a109-05b540f6d706 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:23:47.744754 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2782eb3-a24c-4cb4-a109-05b540f6d706 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:23:47.763255 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-c2782eb3-a24c-4cb4-a109-05b540f6d706 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:23:47.764721 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-c2782eb3-a24c-4cb4-a109-05b540f6d706 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:23:47.828719 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1229/2487] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:23:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:23:47.836714 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ec3f5d-8732-4078-8341-0ffc5b73e2dd None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:23:47.837075 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ec3f5d-8732-4078-8341-0ffc5b73e2dd None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:23:47.837467 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ec3f5d-8732-4078-8341-0ffc5b73e2dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:23:47.857135 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-e4ec3f5d-8732-4078-8341-0ffc5b73e2dd None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:23:47.859077 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-e4ec3f5d-8732-4078-8341-0ffc5b73e2dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:23:47.868431 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1259/2488] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:23:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:23:47.872943 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2782eb3-a24c-4cb4-a109-05b540f6d706 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:23:47.883292 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2782eb3-a24c-4cb4-a109-05b540f6d706 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:23:47.888124 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fcff87-0895-4c0e-9a10-82a85b1a7e9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:23:47.900766 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fcff87-0895-4c0e-9a10-82a85b1a7e9d admin admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:23:47.900766 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fcff87-0895-4c0e-9a10-82a85b1a7e9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:23:47.900766 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fcff87-0895-4c0e-9a10-82a85b1a7e9d admin admin] PATH_INFO: `/v3/domains` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:23:47.900766 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fcff87-0895-4c0e-9a10-82a85b1a7e9d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 05:23:47.900766 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fcff87-0895-4c0e-9a10-82a85b1a7e9d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:23:47.900766 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fcff87-0895-4c0e-9a10-82a85b1a7e9d admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:23:47.905748 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1230/2489] 10.222.0.23 () {58 vars in 1152 bytes} [Sat Jun 18 05:23:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 05:23:47.910438 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ec3f5d-8732-4078-8341-0ffc5b73e2dd None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:23:47.920670 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ec3f5d-8732-4078-8341-0ffc5b73e2dd None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:23:47.925285 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464f00db-d4ff-41e7-acc0-503246fafe20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:23:47.925966 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464f00db-d4ff-41e7-acc0-503246fafe20 admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:23:47.926249 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464f00db-d4ff-41e7-acc0-503246fafe20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:23:47.926526 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464f00db-d4ff-41e7-acc0-503246fafe20 admin admin] PATH_INFO: `/v3/projects` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:23:47.927215 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464f00db-d4ff-41e7-acc0-503246fafe20 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:23:47.929313 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464f00db-d4ff-41e7-acc0-503246fafe20 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:23:47.932733 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:23:48.029126 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1260/2490] 10.222.0.23 () {60 vars in 1152 bytes} [Sat Jun 18 05:23:47 2022] POST /identity/v3/projects => generated 385 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:23:48.030553 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fcff87-0895-4c0e-9a10-82a85b1a7e9d admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:23:48.040937 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fcff87-0895-4c0e-9a10-82a85b1a7e9d admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:23:48.104751 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6ba57f-1409-4c58-b71c-dc12b78c960e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:23:48.105447 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6ba57f-1409-4c58-b71c-dc12b78c960e admin admin] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:23:48.105731 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6ba57f-1409-4c58-b71c-dc12b78c960e admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:23:48.106003 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6ba57f-1409-4c58-b71c-dc12b78c960e admin admin] PATH_INFO: `/v3/users` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:23:48.106741 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6ba57f-1409-4c58-b71c-dc12b78c960e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:23:48.108819 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6ba57f-1409-4c58-b71c-dc12b78c960e admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:23:48.216040 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1231/2491] 10.222.0.23 () {60 vars in 1143 bytes} [Sat Jun 18 05:23:48 2022] POST /identity/v3/users => generated 293 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:23:48.217494 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464f00db-d4ff-41e7-acc0-503246fafe20 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:23:48.228882 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464f00db-d4ff-41e7-acc0-503246fafe20 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:23:48.235211 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c608fa4-9533-4772-ad6f-4cb4467cbb3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:23:48.235896 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c608fa4-9533-4772-ad6f-4cb4467cbb3f admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:23:48.236740 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c608fa4-9533-4772-ad6f-4cb4467cbb3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:23:48.236740 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c608fa4-9533-4772-ad6f-4cb4467cbb3f admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:23:48.237186 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c608fa4-9533-4772-ad6f-4cb4467cbb3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:23:48.239199 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c608fa4-9533-4772-ad6f-4cb4467cbb3f admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:23:48.252813 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1261/2492] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:23:48 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:23:48.257843 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6ba57f-1409-4c58-b71c-dc12b78c960e admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:23:48.269055 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6ba57f-1409-4c58-b71c-dc12b78c960e admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:23:48.275222 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f40497-4d12-429a-a419-9460625b84c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:23:48.275965 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f40497-4d12-429a-a419-9460625b84c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:23:48.276477 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f40497-4d12-429a-a419-9460625b84c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:23:48.276961 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f40497-4d12-429a-a419-9460625b84c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:23:48.277917 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f40497-4d12-429a-a419-9460625b84c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:23:48.280030 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f40497-4d12-429a-a419-9460625b84c2 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:23:48.292741 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1232/2493] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:23:48 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:23:48.298861 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c608fa4-9533-4772-ad6f-4cb4467cbb3f admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:23:48.310523 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c608fa4-9533-4772-ad6f-4cb4467cbb3f admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:23:48.322077 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf49370d-679f-44ae-9023-5007aad36178 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:23:48.323196 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf49370d-679f-44ae-9023-5007aad36178 admin admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:23:48.323370 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf49370d-679f-44ae-9023-5007aad36178 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:23:48.323673 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf49370d-679f-44ae-9023-5007aad36178 admin admin] PATH_INFO: `/v3/projects/81c38b2c1ae4484bbf2d4939470f45e4/users/e5dc9999f6664c23855b66670f7d6a98/roles/cbf5c4c26898413d93e73b46b41046a3` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:23:48.341405 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf49370d-679f-44ae-9023-5007aad36178 admin admin] RBAC: Authorizing `identity:create_grant(project_id=81c38b2c1ae4484bbf2d4939470f45e4, user_id=e5dc9999f6664c23855b66670f7d6a98, role_id=cbf5c4c26898413d93e73b46b41046a3)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:23:48.344547 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf49370d-679f-44ae-9023-5007aad36178 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:23:48.497313 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1262/2494] 10.222.0.23 () {60 vars in 1482 bytes} [Sat Jun 18 05:23:48 2022] PUT /identity/v3/projects/81c38b2c1ae4484bbf2d4939470f45e4/users/e5dc9999f6664c23855b66670f7d6a98/roles/cbf5c4c26898413d93e73b46b41046a3 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:23:48.502810 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9789a246-eccf-427f-b672-eb81bad3a22b None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:23:48.503101 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9789a246-eccf-427f-b672-eb81bad3a22b None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:23:48.503382 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9789a246-eccf-427f-b672-eb81bad3a22b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:23:48.524706 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-9789a246-eccf-427f-b672-eb81bad3a22b None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:23:48.526015 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-9789a246-eccf-427f-b672-eb81bad3a22b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:23:48.613290 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:23:48.623995 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1233/2495] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:23:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:23:48.634170 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf49370d-679f-44ae-9023-5007aad36178 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:23:48.636406 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-cf49370d-679f-44ae-9023-5007aad36178 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:23:48.704013 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf49370d-679f-44ae-9023-5007aad36178 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:23:48.707905 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b497633d-acf9-4f11-8928-d2001d1d3740 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c4d7cde8f4ad9aa479bb09e325db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:23:48.708680 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b497633d-acf9-4f11-8928-d2001d1d3740 service neutron] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:23:48.709538 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b497633d-acf9-4f11-8928-d2001d1d3740 service neutron] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:23:48.709538 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b497633d-acf9-4f11-8928-d2001d1d3740 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:23:48.730406 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b497633d-acf9-4f11-8928-d2001d1d3740 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:23:48.732208 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b497633d-acf9-4f11-8928-d2001d1d3740 service neutron] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:23:48.747243 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1263/2496] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:23:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 18 05:23:51.676729 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9789a246-eccf-427f-b672-eb81bad3a22b None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:23:51.679983 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-9789a246-eccf-427f-b672-eb81bad3a22b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:23:51.697446 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9789a246-eccf-427f-b672-eb81bad3a22b None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:23:51.703634 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb430735-e78b-45b7-8e75-271d282c4b0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:23:51.704857 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb430735-e78b-45b7-8e75-271d282c4b0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:23:51.705244 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb430735-e78b-45b7-8e75-271d282c4b0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:23:51.705656 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb430735-e78b-45b7-8e75-271d282c4b0b admin admin] PATH_INFO: `/v3/users/826caae04ee4421abe88c7539050c0c9` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:23:51.707107 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb430735-e78b-45b7-8e75-271d282c4b0b admin admin] RBAC: Authorizing `identity:delete_user(user_id=826caae04ee4421abe88c7539050c0c9)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:23:51.714803 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb430735-e78b-45b7-8e75-271d282c4b0b admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:23:51.919741 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-eb430735-e78b-45b7-8e75-271d282c4b0b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '826caae04ee4421abe88c7539050c0c9'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:23:51.920655 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-eb430735-e78b-45b7-8e75-271d282c4b0b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '826caae04ee4421abe88c7539050c0c9'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:23:52.016291 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-eb430735-e78b-45b7-8e75-271d282c4b0b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '826caae04ee4421abe88c7539050c0c9'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:23:52.035531 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-eb430735-e78b-45b7-8e75-271d282c4b0b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '826caae04ee4421abe88c7539050c0c9'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:23:52.045193 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-eb430735-e78b-45b7-8e75-271d282c4b0b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '826caae04ee4421abe88c7539050c0c9'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:23:52.057917 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1234/2497] 10.222.0.23 () {58 vars in 1223 bytes} [Sat Jun 18 05:23:51 2022] DELETE /identity/v3/users/826caae04ee4421abe88c7539050c0c9 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:23:52.058795 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f955d3e8-805d-4c50-b258-7c9613bb831f None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:23:52.059281 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f955d3e8-805d-4c50-b258-7c9613bb831f None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:23:52.059369 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f955d3e8-805d-4c50-b258-7c9613bb831f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:23:52.077743 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-f955d3e8-805d-4c50-b258-7c9613bb831f None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:23:52.079588 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-f955d3e8-805d-4c50-b258-7c9613bb831f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:23:52.147195 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1264/2498] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:23:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:23:52.157565 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb430735-e78b-45b7-8e75-271d282c4b0b admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:23:52.158573 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-eb430735-e78b-45b7-8e75-271d282c4b0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:23:52.236600 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb430735-e78b-45b7-8e75-271d282c4b0b admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:23:52.245556 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b8489b-8f39-41fc-bb2f-5a120f165d4b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c4d7cde8f4ad9aa479bb09e325db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:23:52.246573 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b8489b-8f39-41fc-bb2f-5a120f165d4b service neutron] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:23:52.247177 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b8489b-8f39-41fc-bb2f-5a120f165d4b service neutron] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:23:52.247177 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b8489b-8f39-41fc-bb2f-5a120f165d4b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:23:52.259746 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b8489b-8f39-41fc-bb2f-5a120f165d4b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:23:52.264736 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b8489b-8f39-41fc-bb2f-5a120f165d4b service neutron] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:23:52.273489 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1235/2499] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:23:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:23:53.056818 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f955d3e8-805d-4c50-b258-7c9613bb831f None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:23:53.058307 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-f955d3e8-805d-4c50-b258-7c9613bb831f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:23:53.073192 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f955d3e8-805d-4c50-b258-7c9613bb831f None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:23:53.082799 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b757470-8c13-4ea4-a7be-e6239ef8749d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:23:53.082799 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b757470-8c13-4ea4-a7be-e6239ef8749d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:23:53.082799 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b757470-8c13-4ea4-a7be-e6239ef8749d admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:23:53.082799 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b757470-8c13-4ea4-a7be-e6239ef8749d admin admin] PATH_INFO: `/v3/projects/b2d41818afeb408fbccabda446fb4ad5` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:23:53.083112 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b757470-8c13-4ea4-a7be-e6239ef8749d admin admin] RBAC: Authorizing `identity:delete_project(project_id=b2d41818afeb408fbccabda446fb4ad5)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:23:53.085005 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b757470-8c13-4ea4-a7be-e6239ef8749d admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:23:53.232946 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:23:53.234321 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-2b757470-8c13-4ea4-a7be-e6239ef8749d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b2d41818afeb408fbccabda446fb4ad5', 'request_id': 'req-2b757470-8c13-4ea4-a7be-e6239ef8749d'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:23:53.241104 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-2b757470-8c13-4ea4-a7be-e6239ef8749d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b2d41818afeb408fbccabda446fb4ad5', 'request_id': 'req-2b757470-8c13-4ea4-a7be-e6239ef8749d'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:23:53.247954 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-2b757470-8c13-4ea4-a7be-e6239ef8749d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b2d41818afeb408fbccabda446fb4ad5', 'request_id': 'req-2b757470-8c13-4ea4-a7be-e6239ef8749d'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:23:53.333821 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-2b757470-8c13-4ea4-a7be-e6239ef8749d admin admin] The token cache is being invalidate because project b2d41818afeb408fbccabda446fb4ad5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80469) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 18 05:23:53.334438 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-2b757470-8c13-4ea4-a7be-e6239ef8749d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:23:53.334926 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-2b757470-8c13-4ea4-a7be-e6239ef8749d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:23:53.345298 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1265/2500] 10.222.0.23 () {58 vars in 1232 bytes} [Sat Jun 18 05:23:53 2022] DELETE /identity/v3/projects/b2d41818afeb408fbccabda446fb4ad5 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:23:53.350143 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b8489b-8f39-41fc-bb2f-5a120f165d4b service neutron] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:23:53.365028 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b8489b-8f39-41fc-bb2f-5a120f165d4b service neutron] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:23:53.457548 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd275f41-f4d3-4c28-9857-1dd3b8ebde52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:23:53.459298 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd275f41-f4d3-4c28-9857-1dd3b8ebde52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:23:53.459298 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd275f41-f4d3-4c28-9857-1dd3b8ebde52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:23:53.462128 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd275f41-f4d3-4c28-9857-1dd3b8ebde52 admin admin] PATH_INFO: `/v3/users/1964ca0181384d35a53e9bf7505ecba6` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:23:53.462128 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd275f41-f4d3-4c28-9857-1dd3b8ebde52 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1964ca0181384d35a53e9bf7505ecba6)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:23:53.464041 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd275f41-f4d3-4c28-9857-1dd3b8ebde52 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:23:53.485752 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:23:53.673735 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-cd275f41-f4d3-4c28-9857-1dd3b8ebde52 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1964ca0181384d35a53e9bf7505ecba6'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:23:53.674640 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-cd275f41-f4d3-4c28-9857-1dd3b8ebde52 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1964ca0181384d35a53e9bf7505ecba6'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:23:53.773239 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b757470-8c13-4ea4-a7be-e6239ef8749d admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:23:53.774496 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-cd275f41-f4d3-4c28-9857-1dd3b8ebde52 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1964ca0181384d35a53e9bf7505ecba6'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:23:53.776625 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-2b757470-8c13-4ea4-a7be-e6239ef8749d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:23:53.801737 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-cd275f41-f4d3-4c28-9857-1dd3b8ebde52 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1964ca0181384d35a53e9bf7505ecba6'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:23:53.814250 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-cd275f41-f4d3-4c28-9857-1dd3b8ebde52 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1964ca0181384d35a53e9bf7505ecba6'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:23:53.832819 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1236/2501] 10.222.0.23 () {58 vars in 1223 bytes} [Sat Jun 18 05:23:53 2022] DELETE /identity/v3/users/1964ca0181384d35a53e9bf7505ecba6 => generated 0 bytes in 481 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:23:53.898989 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b757470-8c13-4ea4-a7be-e6239ef8749d admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:23:53.961404 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3888f760-416b-4ce9-bacd-f37d15117362 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c4d7cde8f4ad9aa479bb09e325db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:23:53.962242 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3888f760-416b-4ce9-bacd-f37d15117362 service neutron] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:23:53.962592 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3888f760-416b-4ce9-bacd-f37d15117362 service neutron] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:23:53.962941 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3888f760-416b-4ce9-bacd-f37d15117362 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:23:53.965042 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-3888f760-416b-4ce9-bacd-f37d15117362 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:23:54.057715 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3888f760-416b-4ce9-bacd-f37d15117362 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:23:54.059207 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3888f760-416b-4ce9-bacd-f37d15117362 service neutron] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:23:54.072142 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1266/2502] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:23:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:23:54.468631 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd275f41-f4d3-4c28-9857-1dd3b8ebde52 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:23:54.471503 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-cd275f41-f4d3-4c28-9857-1dd3b8ebde52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:23:54.493673 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd275f41-f4d3-4c28-9857-1dd3b8ebde52 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:23:54.506258 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9729cbf-4c92-4dfa-8496-1122877101f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:23:54.507499 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9729cbf-4c92-4dfa-8496-1122877101f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:23:54.508198 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9729cbf-4c92-4dfa-8496-1122877101f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:23:54.508921 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9729cbf-4c92-4dfa-8496-1122877101f0 admin admin] PATH_INFO: `/v3/projects/edf89c88b9864058882d81837bae11f9` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:23:54.510768 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9729cbf-4c92-4dfa-8496-1122877101f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=edf89c88b9864058882d81837bae11f9)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:23:54.512913 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9729cbf-4c92-4dfa-8496-1122877101f0 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:23:54.663576 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-d9729cbf-4c92-4dfa-8496-1122877101f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'edf89c88b9864058882d81837bae11f9', 'request_id': 'req-d9729cbf-4c92-4dfa-8496-1122877101f0'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:23:54.679795 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-d9729cbf-4c92-4dfa-8496-1122877101f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'edf89c88b9864058882d81837bae11f9', 'request_id': 'req-d9729cbf-4c92-4dfa-8496-1122877101f0'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:23:54.760622 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-d9729cbf-4c92-4dfa-8496-1122877101f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'edf89c88b9864058882d81837bae11f9', 'request_id': 'req-d9729cbf-4c92-4dfa-8496-1122877101f0'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:23:54.774022 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e19f71-b7e5-4e22-8b60-761ae306ce17 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:23:54.774558 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e19f71-b7e5-4e22-8b60-761ae306ce17 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:23:54.774663 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e19f71-b7e5-4e22-8b60-761ae306ce17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:23:54.806025 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-d9729cbf-4c92-4dfa-8496-1122877101f0 admin admin] The token cache is being invalidate because project edf89c88b9864058882d81837bae11f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80468) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 18 05:23:54.806630 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-d9729cbf-4c92-4dfa-8496-1122877101f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:23:54.813239 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-d9729cbf-4c92-4dfa-8496-1122877101f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:23:54.813894 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1237/2503] 10.222.0.23 () {58 vars in 1232 bytes} [Sat Jun 18 05:23:54 2022] DELETE /identity/v3/projects/edf89c88b9864058882d81837bae11f9 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:23:54.821569 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-a7e19f71-b7e5-4e22-8b60-761ae306ce17 None None] MFA Rules not processed for user `e5dc9999f6664c23855b66670f7d6a98`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:23:54.822826 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-a7e19f71-b7e5-4e22-8b60-761ae306ce17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:23:54.823447 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98674fe3-fac9-466e-b32a-22d52b5bcf59 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:23:54.823742 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98674fe3-fac9-466e-b32a-22d52b5bcf59 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:23:54.824027 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98674fe3-fac9-466e-b32a-22d52b5bcf59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:23:54.861049 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-98674fe3-fac9-466e-b32a-22d52b5bcf59 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:23:54.862597 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-98674fe3-fac9-466e-b32a-22d52b5bcf59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:23:54.895369 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:23:54.943084 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1238/2504] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:23:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:23:54.955543 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6accf98-b485-4752-94ca-a41ea8ce3fcf None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:23:54.956027 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6accf98-b485-4752-94ca-a41ea8ce3fcf None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:23:54.956387 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6accf98-b485-4752-94ca-a41ea8ce3fcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:23:54.977634 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1267/2505] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:23:54 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:23:54.978699 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-e6accf98-b485-4752-94ca-a41ea8ce3fcf None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:23:54.980327 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-e6accf98-b485-4752-94ca-a41ea8ce3fcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:23:54.987345 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e19f71-b7e5-4e22-8b60-761ae306ce17 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:23:54.989051 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-a7e19f71-b7e5-4e22-8b60-761ae306ce17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:23:54.994786 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1239/2506] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:23:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:23:54.996031 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6accf98-b485-4752-94ca-a41ea8ce3fcf None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:23:55.006532 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6accf98-b485-4752-94ca-a41ea8ce3fcf None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:23:55.012475 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb18ca4-272a-4bdc-ace7-1a1007061285 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:23:55.013262 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb18ca4-272a-4bdc-ace7-1a1007061285 admin admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:23:55.013691 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb18ca4-272a-4bdc-ace7-1a1007061285 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:23:55.014041 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb18ca4-272a-4bdc-ace7-1a1007061285 admin admin] PATH_INFO: `/v3/domains` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:23:55.014880 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb18ca4-272a-4bdc-ace7-1a1007061285 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 05:23:55.015225 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb18ca4-272a-4bdc-ace7-1a1007061285 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:23:55.017394 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb18ca4-272a-4bdc-ace7-1a1007061285 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:23:55.030326 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:23:55.042090 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1240/2507] 10.222.0.23 () {58 vars in 1152 bytes} [Sat Jun 18 05:23:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 05:23:55.043002 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb18ca4-272a-4bdc-ace7-1a1007061285 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:23:55.055340 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb18ca4-272a-4bdc-ace7-1a1007061285 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:23:55.065122 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62420cbe-23ce-427e-ad96-d21ad49b2999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:23:55.065122 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62420cbe-23ce-427e-ad96-d21ad49b2999 admin admin] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:23:55.065122 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62420cbe-23ce-427e-ad96-d21ad49b2999 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:23:55.065122 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62420cbe-23ce-427e-ad96-d21ad49b2999 admin admin] PATH_INFO: `/v3/projects` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:23:55.065567 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62420cbe-23ce-427e-ad96-d21ad49b2999 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:23:55.065567 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62420cbe-23ce-427e-ad96-d21ad49b2999 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:23:55.085790 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e19f71-b7e5-4e22-8b60-761ae306ce17 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:23:55.091586 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3ab1c5-0cdf-434e-8a42-fe18a2d70827 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15e7f45ee0744c1d9602187eac8e8b80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:23:55.092258 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3ab1c5-0cdf-434e-8a42-fe18a2d70827 service nova] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:23:55.092542 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3ab1c5-0cdf-434e-8a42-fe18a2d70827 service nova] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:23:55.092981 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3ab1c5-0cdf-434e-8a42-fe18a2d70827 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:23:55.102402 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3ab1c5-0cdf-434e-8a42-fe18a2d70827 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:23:55.104197 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3ab1c5-0cdf-434e-8a42-fe18a2d70827 service nova] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:23:55.114458 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1268/2508] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:23:54 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 18 05:23:55.188997 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1241/2509] 10.222.0.23 () {60 vars in 1152 bytes} [Sat Jun 18 05:23:55 2022] POST /identity/v3/projects => generated 387 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:23:55.194023 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3ab1c5-0cdf-434e-8a42-fe18a2d70827 service nova] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:23:55.205576 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3ab1c5-0cdf-434e-8a42-fe18a2d70827 service nova] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:23:55.263786 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950ae70f-34ee-4297-aa30-feae56312846 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:23:55.264463 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950ae70f-34ee-4297-aa30-feae56312846 admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:23:55.264746 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950ae70f-34ee-4297-aa30-feae56312846 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:23:55.265083 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950ae70f-34ee-4297-aa30-feae56312846 admin admin] PATH_INFO: `/v3/users` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:23:55.265842 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950ae70f-34ee-4297-aa30-feae56312846 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:23:55.267859 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950ae70f-34ee-4297-aa30-feae56312846 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:23:55.399046 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1269/2510] 10.222.0.23 () {60 vars in 1143 bytes} [Sat Jun 18 05:23:55 2022] POST /identity/v3/users => generated 294 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:23:55.400106 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62420cbe-23ce-427e-ad96-d21ad49b2999 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:23:55.418335 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62420cbe-23ce-427e-ad96-d21ad49b2999 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:23:55.426048 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05eed110-9853-4523-b66a-aa952cf8d0d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:23:55.426977 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05eed110-9853-4523-b66a-aa952cf8d0d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:23:55.427794 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05eed110-9853-4523-b66a-aa952cf8d0d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:23:55.428522 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05eed110-9853-4523-b66a-aa952cf8d0d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:23:55.429348 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05eed110-9853-4523-b66a-aa952cf8d0d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:23:55.431094 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05eed110-9853-4523-b66a-aa952cf8d0d4 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:23:55.443890 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1242/2511] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:23:55 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:23:55.447680 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950ae70f-34ee-4297-aa30-feae56312846 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:23:55.462129 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950ae70f-34ee-4297-aa30-feae56312846 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:23:55.473242 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e10a5a-e852-485c-b8f5-78b5d659d2ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:23:55.477018 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e10a5a-e852-485c-b8f5-78b5d659d2ae admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:23:55.477310 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e10a5a-e852-485c-b8f5-78b5d659d2ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:23:55.477623 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e10a5a-e852-485c-b8f5-78b5d659d2ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:23:55.478399 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e10a5a-e852-485c-b8f5-78b5d659d2ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:23:55.480035 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e10a5a-e852-485c-b8f5-78b5d659d2ae admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:23:55.500767 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1270/2512] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:23:55 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:23:55.507528 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05eed110-9853-4523-b66a-aa952cf8d0d4 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:23:55.522527 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05eed110-9853-4523-b66a-aa952cf8d0d4 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:23:55.535843 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabf4c82-c582-46f3-8c18-127e38a09e4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:23:55.536393 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabf4c82-c582-46f3-8c18-127e38a09e4f admin admin] REQUEST_METHOD: `PUT` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:23:55.537270 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabf4c82-c582-46f3-8c18-127e38a09e4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:23:55.537635 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabf4c82-c582-46f3-8c18-127e38a09e4f admin admin] PATH_INFO: `/v3/projects/bf2f942e811d4bcc8b5d80d504653384/users/b6532f9aec0a46778681746a15e86d2d/roles/cbf5c4c26898413d93e73b46b41046a3` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:23:55.565462 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabf4c82-c582-46f3-8c18-127e38a09e4f admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf2f942e811d4bcc8b5d80d504653384, user_id=b6532f9aec0a46778681746a15e86d2d, role_id=cbf5c4c26898413d93e73b46b41046a3)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:23:55.568870 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabf4c82-c582-46f3-8c18-127e38a09e4f admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:23:55.678219 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1243/2513] 10.222.0.23 () {60 vars in 1482 bytes} [Sat Jun 18 05:23:55 2022] PUT /identity/v3/projects/bf2f942e811d4bcc8b5d80d504653384/users/b6532f9aec0a46778681746a15e86d2d/roles/cbf5c4c26898413d93e73b46b41046a3 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:23:55.687985 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7157c68-994f-4c2b-8f8b-fa51983976b6 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:23:55.688453 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7157c68-994f-4c2b-8f8b-fa51983976b6 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:23:55.688902 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7157c68-994f-4c2b-8f8b-fa51983976b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:23:55.746766 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-f7157c68-994f-4c2b-8f8b-fa51983976b6 None None] MFA Rules not processed for user `b6532f9aec0a46778681746a15e86d2d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:23:55.747823 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-f7157c68-994f-4c2b-8f8b-fa51983976b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:23:55.776761 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:23:55.834479 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabf4c82-c582-46f3-8c18-127e38a09e4f admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:23:55.835030 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1271/2514] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:23:55 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:23:55.932083 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabf4c82-c582-46f3-8c18-127e38a09e4f admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:23:55.937630 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb8ff85-f6bf-4fb1-a86c-2dbb3e73da95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:23:55.938376 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb8ff85-f6bf-4fb1-a86c-2dbb3e73da95 admin admin] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:23:55.938664 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb8ff85-f6bf-4fb1-a86c-2dbb3e73da95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:23:55.938944 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb8ff85-f6bf-4fb1-a86c-2dbb3e73da95 admin admin] PATH_INFO: `/v3/projects` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:23:55.939695 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb8ff85-f6bf-4fb1-a86c-2dbb3e73da95 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:23:55.942095 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb8ff85-f6bf-4fb1-a86c-2dbb3e73da95 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:23:56.046253 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1244/2515] 10.222.0.23 () {60 vars in 1152 bytes} [Sat Jun 18 05:23:55 2022] POST /identity/v3/projects => generated 385 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:23:56.050274 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7157c68-994f-4c2b-8f8b-fa51983976b6 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:23:56.070178 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7157c68-994f-4c2b-8f8b-fa51983976b6 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:23:56.166909 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816365f5-b1be-4dcf-8e4f-387f2ea4a3c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:23:56.169550 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816365f5-b1be-4dcf-8e4f-387f2ea4a3c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:23:56.169962 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816365f5-b1be-4dcf-8e4f-387f2ea4a3c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:23:56.170256 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816365f5-b1be-4dcf-8e4f-387f2ea4a3c6 admin admin] PATH_INFO: `/v3/users` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:23:56.176899 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816365f5-b1be-4dcf-8e4f-387f2ea4a3c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:23:56.177933 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816365f5-b1be-4dcf-8e4f-387f2ea4a3c6 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:23:56.302323 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1272/2516] 10.222.0.23 () {60 vars in 1143 bytes} [Sat Jun 18 05:23:56 2022] POST /identity/v3/users => generated 299 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:23:56.306803 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb8ff85-f6bf-4fb1-a86c-2dbb3e73da95 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:23:56.319242 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb8ff85-f6bf-4fb1-a86c-2dbb3e73da95 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:23:56.324044 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a3d9ed-17ab-4be2-a654-f885fee331a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:23:56.327053 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a3d9ed-17ab-4be2-a654-f885fee331a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:23:56.327053 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a3d9ed-17ab-4be2-a654-f885fee331a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:23:56.327053 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a3d9ed-17ab-4be2-a654-f885fee331a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:23:56.327053 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a3d9ed-17ab-4be2-a654-f885fee331a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:23:56.327636 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a3d9ed-17ab-4be2-a654-f885fee331a3 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:23:56.358607 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1245/2517] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:23:56 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:23:56.362538 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816365f5-b1be-4dcf-8e4f-387f2ea4a3c6 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:23:56.379238 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816365f5-b1be-4dcf-8e4f-387f2ea4a3c6 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:23:56.393368 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bc7c8c-5d55-4b7b-9004-5a20a383a92d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:23:56.394771 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bc7c8c-5d55-4b7b-9004-5a20a383a92d admin admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:23:56.395283 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bc7c8c-5d55-4b7b-9004-5a20a383a92d admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:23:56.395665 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bc7c8c-5d55-4b7b-9004-5a20a383a92d admin admin] PATH_INFO: `/v3/projects/f96a4371b173477cb1b899490873dba0/users/69b9f11a98b84bd48185b369109fc6ff/roles/21c65f2c51bd4dd283d13507d2714467` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:23:56.416145 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bc7c8c-5d55-4b7b-9004-5a20a383a92d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f96a4371b173477cb1b899490873dba0, user_id=69b9f11a98b84bd48185b369109fc6ff, role_id=21c65f2c51bd4dd283d13507d2714467)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:23:56.419372 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bc7c8c-5d55-4b7b-9004-5a20a383a92d admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:23:56.493769 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1273/2518] 10.222.0.23 () {60 vars in 1482 bytes} [Sat Jun 18 05:23:56 2022] PUT /identity/v3/projects/f96a4371b173477cb1b899490873dba0/users/69b9f11a98b84bd48185b369109fc6ff/roles/21c65f2c51bd4dd283d13507d2714467 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:23:56.497466 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a3d9ed-17ab-4be2-a654-f885fee331a3 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:23:56.508086 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a3d9ed-17ab-4be2-a654-f885fee331a3 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:23:56.592756 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7b760d-c767-4479-ab30-4ba00ce9aa10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:23:56.593746 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7b760d-c767-4479-ab30-4ba00ce9aa10 admin admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:23:56.594421 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7b760d-c767-4479-ab30-4ba00ce9aa10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:23:56.594876 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7b760d-c767-4479-ab30-4ba00ce9aa10 admin admin] PATH_INFO: `/v3/roles` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:23:56.595765 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7b760d-c767-4479-ab30-4ba00ce9aa10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:23:56.597823 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7b760d-c767-4479-ab30-4ba00ce9aa10 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:23:56.610411 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1246/2519] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:23:56 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 18 05:23:56.614219 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bc7c8c-5d55-4b7b-9004-5a20a383a92d admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:23:56.636971 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bc7c8c-5d55-4b7b-9004-5a20a383a92d admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:23:56.643491 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d10ecc-4340-42ed-8cf2-0d0109718798 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:23:56.644660 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d10ecc-4340-42ed-8cf2-0d0109718798 admin admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:23:56.645150 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d10ecc-4340-42ed-8cf2-0d0109718798 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:23:56.645475 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d10ecc-4340-42ed-8cf2-0d0109718798 admin admin] PATH_INFO: `/v3/projects/f96a4371b173477cb1b899490873dba0/users/69b9f11a98b84bd48185b369109fc6ff/roles/21c65f2c51bd4dd283d13507d2714467` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:23:56.648294 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d10ecc-4340-42ed-8cf2-0d0109718798 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f96a4371b173477cb1b899490873dba0, user_id=69b9f11a98b84bd48185b369109fc6ff, role_id=21c65f2c51bd4dd283d13507d2714467)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:23:56.651062 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d10ecc-4340-42ed-8cf2-0d0109718798 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:23:56.675542 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1274/2520] 10.222.0.23 () {60 vars in 1482 bytes} [Sat Jun 18 05:23:56 2022] PUT /identity/v3/projects/f96a4371b173477cb1b899490873dba0/users/69b9f11a98b84bd48185b369109fc6ff/roles/21c65f2c51bd4dd283d13507d2714467 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:23:56.750287 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d0b1a9-1e26-4ec9-aead-1f61ef891c86 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:23:56.750797 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d0b1a9-1e26-4ec9-aead-1f61ef891c86 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:23:56.751493 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d0b1a9-1e26-4ec9-aead-1f61ef891c86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:23:56.775942 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:23:56.785044 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-a7d0b1a9-1e26-4ec9-aead-1f61ef891c86 None None] MFA Rules not processed for user `69b9f11a98b84bd48185b369109fc6ff`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:23:56.786412 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-a7d0b1a9-1e26-4ec9-aead-1f61ef891c86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:23:56.885321 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1247/2521] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:23:56 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:23:56.896831 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d10ecc-4340-42ed-8cf2-0d0109718798 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:23:56.898932 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-06d10ecc-4340-42ed-8cf2-0d0109718798 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:23:56.965611 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d10ecc-4340-42ed-8cf2-0d0109718798 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:23:56.975303 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b269d67-c999-4c8c-acb3-c1f1825da814 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9de0428abc8400aa338aba4f7c34c94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:23:56.975459 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b269d67-c999-4c8c-acb3-c1f1825da814 service cinder] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:23:56.975789 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b269d67-c999-4c8c-acb3-c1f1825da814 service cinder] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:23:56.976203 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b269d67-c999-4c8c-acb3-c1f1825da814 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:23:56.989364 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b269d67-c999-4c8c-acb3-c1f1825da814 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:23:56.990954 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b269d67-c999-4c8c-acb3-c1f1825da814 service cinder] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:23:57.002564 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1275/2522] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:23:56 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:23:58.695073 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d0b1a9-1e26-4ec9-aead-1f61ef891c86 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:23:58.697090 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a65ccc6-6440-4620-b373-f1cb2ee82afa None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:23:58.697504 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a65ccc6-6440-4620-b373-f1cb2ee82afa None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:23:58.697759 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a65ccc6-6440-4620-b373-f1cb2ee82afa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:23:58.736679 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-2a65ccc6-6440-4620-b373-f1cb2ee82afa None None] MFA Rules not processed for user `e5dc9999f6664c23855b66670f7d6a98`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:23:58.738423 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-2a65ccc6-6440-4620-b373-f1cb2ee82afa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:23:58.769550 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:23:58.786871 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d0b1a9-1e26-4ec9-aead-1f61ef891c86 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:23:58.793126 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68dcd03-9d23-4738-9c54-894b8f70fc5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:23:58.794453 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68dcd03-9d23-4738-9c54-894b8f70fc5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:23:58.795009 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68dcd03-9d23-4738-9c54-894b8f70fc5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:23:58.795930 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68dcd03-9d23-4738-9c54-894b8f70fc5c admin admin] PATH_INFO: `/v3/users/b6532f9aec0a46778681746a15e86d2d` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:23:58.798280 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68dcd03-9d23-4738-9c54-894b8f70fc5c admin admin] RBAC: Authorizing `identity:delete_user(user_id=b6532f9aec0a46778681746a15e86d2d)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:23:58.800957 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68dcd03-9d23-4738-9c54-894b8f70fc5c admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:23:58.862228 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1276/2523] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:23:58 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 18 05:23:59.076555 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-a68dcd03-9d23-4738-9c54-894b8f70fc5c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b6532f9aec0a46778681746a15e86d2d'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:23:59.077656 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-a68dcd03-9d23-4738-9c54-894b8f70fc5c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b6532f9aec0a46778681746a15e86d2d'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:23:59.179235 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-a68dcd03-9d23-4738-9c54-894b8f70fc5c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b6532f9aec0a46778681746a15e86d2d'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:23:59.182223 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a65ccc6-6440-4620-b373-f1cb2ee82afa None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:23:59.183828 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-2a65ccc6-6440-4620-b373-f1cb2ee82afa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:23:59.206365 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-a68dcd03-9d23-4738-9c54-894b8f70fc5c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b6532f9aec0a46778681746a15e86d2d'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:23:59.212971 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a65ccc6-6440-4620-b373-f1cb2ee82afa None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:23:59.214076 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0920a3d-9ff9-4c9f-8ebd-654888a361d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:23:59.214957 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0920a3d-9ff9-4c9f-8ebd-654888a361d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:23:59.215236 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0920a3d-9ff9-4c9f-8ebd-654888a361d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:23:59.215508 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0920a3d-9ff9-4c9f-8ebd-654888a361d9 admin admin] PATH_INFO: `/v3/users/e0f3999a062e45748878a3204e3ea456` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:23:59.217529 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0920a3d-9ff9-4c9f-8ebd-654888a361d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e0f3999a062e45748878a3204e3ea456)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:23:59.220071 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0920a3d-9ff9-4c9f-8ebd-654888a361d9 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:23:59.222553 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-a68dcd03-9d23-4738-9c54-894b8f70fc5c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b6532f9aec0a46778681746a15e86d2d'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:23:59.232049 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1248/2524] 10.222.0.23 () {58 vars in 1223 bytes} [Sat Jun 18 05:23:58 2022] DELETE /identity/v3/users/b6532f9aec0a46778681746a15e86d2d => generated 0 bytes in 540 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:23:59.236754 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fedc92-6b54-41f1-b1a2-69966283af86 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:23:59.237046 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fedc92-6b54-41f1-b1a2-69966283af86 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:23:59.237467 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fedc92-6b54-41f1-b1a2-69966283af86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:23:59.265223 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-03fedc92-6b54-41f1-b1a2-69966283af86 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:23:59.266511 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-03fedc92-6b54-41f1-b1a2-69966283af86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:23:59.278029 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:23:59.345923 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1249/2525] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:23:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:23:59.356259 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fedc92-6b54-41f1-b1a2-69966283af86 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:23:59.357754 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-03fedc92-6b54-41f1-b1a2-69966283af86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:23:59.423729 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-b0920a3d-9ff9-4c9f-8ebd-654888a361d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e0f3999a062e45748878a3204e3ea456'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:23:59.453133 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-b0920a3d-9ff9-4c9f-8ebd-654888a361d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e0f3999a062e45748878a3204e3ea456'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:23:59.455695 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-b0920a3d-9ff9-4c9f-8ebd-654888a361d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e0f3999a062e45748878a3204e3ea456'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:23:59.475319 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fedc92-6b54-41f1-b1a2-69966283af86 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:23:59.481341 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17db988f-73aa-404b-a386-22aff4619b3c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c4d7cde8f4ad9aa479bb09e325db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:23:59.482169 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17db988f-73aa-404b-a386-22aff4619b3c service neutron] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:23:59.482838 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17db988f-73aa-404b-a386-22aff4619b3c service neutron] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:23:59.482943 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17db988f-73aa-404b-a386-22aff4619b3c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:23:59.486552 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-17db988f-73aa-404b-a386-22aff4619b3c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:23:59.501424 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17db988f-73aa-404b-a386-22aff4619b3c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:23:59.503204 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17db988f-73aa-404b-a386-22aff4619b3c service neutron] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:23:59.516271 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1250/2526] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:23:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 18 05:23:59.530620 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-b0920a3d-9ff9-4c9f-8ebd-654888a361d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e0f3999a062e45748878a3204e3ea456'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:23:59.543504 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-b0920a3d-9ff9-4c9f-8ebd-654888a361d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e0f3999a062e45748878a3204e3ea456'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:23:59.555771 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1277/2527] 10.222.0.23 () {58 vars in 1223 bytes} [Sat Jun 18 05:23:59 2022] DELETE /identity/v3/users/e0f3999a062e45748878a3204e3ea456 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 18 05:23:59.907399 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17db988f-73aa-404b-a386-22aff4619b3c service neutron] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:23:59.909947 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-17db988f-73aa-404b-a386-22aff4619b3c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:23:59.925513 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17db988f-73aa-404b-a386-22aff4619b3c service neutron] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:24:00.011845 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d489f8-0fe5-4344-ae15-0221b13d5491 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:24:00.013077 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d489f8-0fe5-4344-ae15-0221b13d5491 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:24:00.013659 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d489f8-0fe5-4344-ae15-0221b13d5491 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:24:00.014109 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d489f8-0fe5-4344-ae15-0221b13d5491 admin admin] PATH_INFO: `/v3/projects/758883a5aea04c85ad65bb6970bdb098` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:24:00.015652 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d489f8-0fe5-4344-ae15-0221b13d5491 admin admin] RBAC: Authorizing `identity:delete_project(project_id=758883a5aea04c85ad65bb6970bdb098)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:24:00.017493 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d489f8-0fe5-4344-ae15-0221b13d5491 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:24:00.024818 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0920a3d-9ff9-4c9f-8ebd-654888a361d9 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:24:00.027478 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-b0920a3d-9ff9-4c9f-8ebd-654888a361d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:24:00.058285 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0920a3d-9ff9-4c9f-8ebd-654888a361d9 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:24:00.066342 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70249df0-c5d7-4b78-89ad-bbe82edeb09d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:24:00.067441 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70249df0-c5d7-4b78-89ad-bbe82edeb09d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:24:00.067756 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70249df0-c5d7-4b78-89ad-bbe82edeb09d admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:24:00.068063 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70249df0-c5d7-4b78-89ad-bbe82edeb09d admin admin] PATH_INFO: `/v3/projects/bf2f942e811d4bcc8b5d80d504653384` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:24:00.069402 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70249df0-c5d7-4b78-89ad-bbe82edeb09d admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf2f942e811d4bcc8b5d80d504653384)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:24:00.071454 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70249df0-c5d7-4b78-89ad-bbe82edeb09d admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:24:00.162787 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-36d489f8-0fe5-4344-ae15-0221b13d5491 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '758883a5aea04c85ad65bb6970bdb098', 'request_id': 'req-36d489f8-0fe5-4344-ae15-0221b13d5491'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:24:00.168904 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-36d489f8-0fe5-4344-ae15-0221b13d5491 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '758883a5aea04c85ad65bb6970bdb098', 'request_id': 'req-36d489f8-0fe5-4344-ae15-0221b13d5491'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:24:00.171670 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:24:00.209063 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-70249df0-c5d7-4b78-89ad-bbe82edeb09d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf2f942e811d4bcc8b5d80d504653384', 'request_id': 'req-70249df0-c5d7-4b78-89ad-bbe82edeb09d'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:24:00.217407 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-70249df0-c5d7-4b78-89ad-bbe82edeb09d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf2f942e811d4bcc8b5d80d504653384', 'request_id': 'req-70249df0-c5d7-4b78-89ad-bbe82edeb09d'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:24:00.222382 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-70249df0-c5d7-4b78-89ad-bbe82edeb09d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf2f942e811d4bcc8b5d80d504653384', 'request_id': 'req-70249df0-c5d7-4b78-89ad-bbe82edeb09d'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:24:00.274549 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-36d489f8-0fe5-4344-ae15-0221b13d5491 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '758883a5aea04c85ad65bb6970bdb098', 'request_id': 'req-36d489f8-0fe5-4344-ae15-0221b13d5491'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:24:00.287514 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-36d489f8-0fe5-4344-ae15-0221b13d5491 admin admin] The token cache is being invalidate because project 758883a5aea04c85ad65bb6970bdb098 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80468) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 18 05:24:00.287820 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-36d489f8-0fe5-4344-ae15-0221b13d5491 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:24:00.288454 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-36d489f8-0fe5-4344-ae15-0221b13d5491 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:24:00.290113 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1251/2528] 10.222.0.23 () {58 vars in 1232 bytes} [Sat Jun 18 05:23:59 2022] DELETE /identity/v3/projects/758883a5aea04c85ad65bb6970bdb098 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:24:00.294903 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb984f8b-a824-421c-8677-5c3253dc37a5 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:24:00.295191 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb984f8b-a824-421c-8677-5c3253dc37a5 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:24:00.295540 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb984f8b-a824-421c-8677-5c3253dc37a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:24:00.315629 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-cb984f8b-a824-421c-8677-5c3253dc37a5 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:24:00.316886 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-cb984f8b-a824-421c-8677-5c3253dc37a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:24:00.370918 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-70249df0-c5d7-4b78-89ad-bbe82edeb09d admin admin] The token cache is being invalidate because project bf2f942e811d4bcc8b5d80d504653384 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80469) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 18 05:24:00.371208 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-70249df0-c5d7-4b78-89ad-bbe82edeb09d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:24:00.371892 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-70249df0-c5d7-4b78-89ad-bbe82edeb09d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:24:00.373714 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1278/2529] 10.222.0.23 () {58 vars in 1232 bytes} [Sat Jun 18 05:24:00 2022] DELETE /identity/v3/projects/bf2f942e811d4bcc8b5d80d504653384 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 18 05:24:00.377509 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70249df0-c5d7-4b78-89ad-bbe82edeb09d admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:24:00.378963 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-70249df0-c5d7-4b78-89ad-bbe82edeb09d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:24:00.388404 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1252/2530] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:24:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 18 05:24:00.396710 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3507b2be-dfae-4a61-a65c-08f94efc7139 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:24:00.397008 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3507b2be-dfae-4a61-a65c-08f94efc7139 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:24:00.397416 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3507b2be-dfae-4a61-a65c-08f94efc7139 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:24:00.411090 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:24:00.418187 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-3507b2be-dfae-4a61-a65c-08f94efc7139 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:24:00.419601 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-3507b2be-dfae-4a61-a65c-08f94efc7139 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:24:00.428504 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1253/2531] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:24:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:24:00.432796 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3507b2be-dfae-4a61-a65c-08f94efc7139 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:24:00.442026 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3507b2be-dfae-4a61-a65c-08f94efc7139 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:24:00.447272 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8845803-92d1-47d2-af66-96f6f7c2fcd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:24:00.447928 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8845803-92d1-47d2-af66-96f6f7c2fcd0 admin admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:24:00.448249 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8845803-92d1-47d2-af66-96f6f7c2fcd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:24:00.448669 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8845803-92d1-47d2-af66-96f6f7c2fcd0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:24:00.449470 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8845803-92d1-47d2-af66-96f6f7c2fcd0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 05:24:00.449796 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8845803-92d1-47d2-af66-96f6f7c2fcd0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:24:00.451433 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8845803-92d1-47d2-af66-96f6f7c2fcd0 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:24:00.456865 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70249df0-c5d7-4b78-89ad-bbe82edeb09d admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:24:00.468260 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2871adac-ac6c-4eab-a94b-9c9ebee019ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:24:00.468260 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2871adac-ac6c-4eab-a94b-9c9ebee019ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:24:00.468260 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2871adac-ac6c-4eab-a94b-9c9ebee019ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:24:00.468260 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2871adac-ac6c-4eab-a94b-9c9ebee019ba admin admin] PATH_INFO: `/v3/users/69b9f11a98b84bd48185b369109fc6ff` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:24:00.468712 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1254/2532] 10.222.0.23 () {58 vars in 1152 bytes} [Sat Jun 18 05:24:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 18 05:24:00.468801 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2871adac-ac6c-4eab-a94b-9c9ebee019ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=69b9f11a98b84bd48185b369109fc6ff)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:24:00.468801 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2871adac-ac6c-4eab-a94b-9c9ebee019ba admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:24:00.468951 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8845803-92d1-47d2-af66-96f6f7c2fcd0 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:24:00.479015 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8845803-92d1-47d2-af66-96f6f7c2fcd0 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:24:00.483817 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ed4cfe-c5e6-4036-8ccc-fce422b60d8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:24:00.484947 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ed4cfe-c5e6-4036-8ccc-fce422b60d8c admin admin] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:24:00.484947 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ed4cfe-c5e6-4036-8ccc-fce422b60d8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:24:00.486175 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ed4cfe-c5e6-4036-8ccc-fce422b60d8c admin admin] PATH_INFO: `/v3/projects` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:24:00.486980 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ed4cfe-c5e6-4036-8ccc-fce422b60d8c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:24:00.488931 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ed4cfe-c5e6-4036-8ccc-fce422b60d8c admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:24:00.598703 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1255/2533] 10.222.0.23 () {60 vars in 1152 bytes} [Sat Jun 18 05:24:00 2022] POST /identity/v3/projects => generated 371 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 18 05:24:00.600160 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ed4cfe-c5e6-4036-8ccc-fce422b60d8c admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:24:00.664612 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ed4cfe-c5e6-4036-8ccc-fce422b60d8c admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:24:00.669648 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ba6de1-2afa-4e0d-9750-16d375fe36a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:24:00.670311 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ba6de1-2afa-4e0d-9750-16d375fe36a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:24:00.670594 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ba6de1-2afa-4e0d-9750-16d375fe36a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:24:00.670871 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ba6de1-2afa-4e0d-9750-16d375fe36a4 admin admin] PATH_INFO: `/v3/users` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:24:00.671563 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ba6de1-2afa-4e0d-9750-16d375fe36a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:24:00.673571 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ba6de1-2afa-4e0d-9750-16d375fe36a4 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:24:00.688908 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-2871adac-ac6c-4eab-a94b-9c9ebee019ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '69b9f11a98b84bd48185b369109fc6ff'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:24:00.700872 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-2871adac-ac6c-4eab-a94b-9c9ebee019ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '69b9f11a98b84bd48185b369109fc6ff'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:24:00.701849 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-2871adac-ac6c-4eab-a94b-9c9ebee019ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '69b9f11a98b84bd48185b369109fc6ff'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:24:00.772683 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1256/2534] 10.222.0.23 () {60 vars in 1143 bytes} [Sat Jun 18 05:24:00 2022] POST /identity/v3/users => generated 286 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 18 05:24:00.776694 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ba6de1-2afa-4e0d-9750-16d375fe36a4 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:24:00.778444 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-e7ba6de1-2afa-4e0d-9750-16d375fe36a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:24:00.790810 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ba6de1-2afa-4e0d-9750-16d375fe36a4 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:24:00.795457 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e90bba0-ffbc-4789-a5f8-69a93d71bc0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:24:00.796093 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e90bba0-ffbc-4789-a5f8-69a93d71bc0d admin admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:24:00.796377 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e90bba0-ffbc-4789-a5f8-69a93d71bc0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:24:00.796654 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e90bba0-ffbc-4789-a5f8-69a93d71bc0d admin admin] PATH_INFO: `/v3/roles` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:24:00.797362 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e90bba0-ffbc-4789-a5f8-69a93d71bc0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:24:00.799004 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e90bba0-ffbc-4789-a5f8-69a93d71bc0d admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:24:00.808803 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1257/2535] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:24:00 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 18 05:24:00.812470 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e90bba0-ffbc-4789-a5f8-69a93d71bc0d admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:24:00.821553 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-2871adac-ac6c-4eab-a94b-9c9ebee019ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '69b9f11a98b84bd48185b369109fc6ff'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:24:00.823328 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e90bba0-ffbc-4789-a5f8-69a93d71bc0d admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:24:00.828334 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72639d5-c604-4307-8fc1-4edeba3cc949 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:24:00.830711 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-2871adac-ac6c-4eab-a94b-9c9ebee019ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '69b9f11a98b84bd48185b369109fc6ff'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:24:00.830874 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72639d5-c604-4307-8fc1-4edeba3cc949 admin admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:24:00.830874 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72639d5-c604-4307-8fc1-4edeba3cc949 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:24:00.830874 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72639d5-c604-4307-8fc1-4edeba3cc949 admin admin] PATH_INFO: `/v3/roles` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:24:00.830874 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72639d5-c604-4307-8fc1-4edeba3cc949 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:24:00.831824 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72639d5-c604-4307-8fc1-4edeba3cc949 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:24:00.835783 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1279/2536] 10.222.0.23 () {58 vars in 1223 bytes} [Sat Jun 18 05:24:00 2022] DELETE /identity/v3/users/69b9f11a98b84bd48185b369109fc6ff => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 18 05:24:00.850756 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1258/2537] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:24:00 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 18 05:24:00.855267 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2871adac-ac6c-4eab-a94b-9c9ebee019ba admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:24:00.920536 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2871adac-ac6c-4eab-a94b-9c9ebee019ba admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:24:00.925322 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461db04a-4a6e-485c-befb-8cd3b01a5c35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:24:00.926548 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461db04a-4a6e-485c-befb-8cd3b01a5c35 admin admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:24:00.926548 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461db04a-4a6e-485c-befb-8cd3b01a5c35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:24:00.926759 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461db04a-4a6e-485c-befb-8cd3b01a5c35 admin admin] PATH_INFO: `/v3/projects/1e2a179f1f5e4593b295d282e34094a8/users/cc3cd4819cbf438fb2c5d4e9edb55360/roles/cbf5c4c26898413d93e73b46b41046a3` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:24:00.942424 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461db04a-4a6e-485c-befb-8cd3b01a5c35 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e2a179f1f5e4593b295d282e34094a8, user_id=cc3cd4819cbf438fb2c5d4e9edb55360, role_id=cbf5c4c26898413d93e73b46b41046a3)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:24:00.945771 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461db04a-4a6e-485c-befb-8cd3b01a5c35 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:24:01.033679 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1280/2538] 10.222.0.23 () {60 vars in 1482 bytes} [Sat Jun 18 05:24:00 2022] PUT /identity/v3/projects/1e2a179f1f5e4593b295d282e34094a8/users/cc3cd4819cbf438fb2c5d4e9edb55360/roles/cbf5c4c26898413d93e73b46b41046a3 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:24:01.035459 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac043ca-0a16-4c74-a70b-2ee745480d55 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:24:01.036178 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac043ca-0a16-4c74-a70b-2ee745480d55 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:24:01.036178 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac043ca-0a16-4c74-a70b-2ee745480d55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:24:01.149243 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-dac043ca-0a16-4c74-a70b-2ee745480d55 None None] MFA Rules not processed for user `cc3cd4819cbf438fb2c5d4e9edb55360`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:24:01.149243 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-dac043ca-0a16-4c74-a70b-2ee745480d55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:24:01.149243 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:DELETE=1,keystone:SELECT=95,keystone:INSERT=4 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:24:01.149243 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1259/2539] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:24:01 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 18 05:24:01.152875 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461db04a-4a6e-485c-befb-8cd3b01a5c35 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:24:01.235474 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461db04a-4a6e-485c-befb-8cd3b01a5c35 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:24:01.251473 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4536998-4a0b-422f-bbf2-f2398262572e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:24:01.251473 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4536998-4a0b-422f-bbf2-f2398262572e admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:24:01.251473 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4536998-4a0b-422f-bbf2-f2398262572e admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:24:01.251473 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4536998-4a0b-422f-bbf2-f2398262572e admin admin] PATH_INFO: `/v3/projects` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:24:01.251955 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4536998-4a0b-422f-bbf2-f2398262572e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:24:01.251955 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4536998-4a0b-422f-bbf2-f2398262572e admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:24:01.526262 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:24:01.575979 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac043ca-0a16-4c74-a70b-2ee745480d55 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:24:01.575979 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-dac043ca-0a16-4c74-a70b-2ee745480d55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:24:01.580285 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1281/2540] 10.222.0.23 () {60 vars in 1152 bytes} [Sat Jun 18 05:24:01 2022] POST /identity/v3/projects => generated 371 bytes in 425 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:24:01.582736 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4536998-4a0b-422f-bbf2-f2398262572e admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:24:01.590169 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac043ca-0a16-4c74-a70b-2ee745480d55 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:24:01.648847 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee017a4-c069-41e3-91b6-fb8718b6d532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:24:01.649742 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee017a4-c069-41e3-91b6-fb8718b6d532 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:24:01.650133 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee017a4-c069-41e3-91b6-fb8718b6d532 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:24:01.650556 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee017a4-c069-41e3-91b6-fb8718b6d532 admin admin] PATH_INFO: `/v3/projects/f96a4371b173477cb1b899490873dba0` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:24:01.652059 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee017a4-c069-41e3-91b6-fb8718b6d532 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f96a4371b173477cb1b899490873dba0)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:24:01.654618 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee017a4-c069-41e3-91b6-fb8718b6d532 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:24:01.669795 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4536998-4a0b-422f-bbf2-f2398262572e admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:24:01.675761 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8243e982-4dc9-4431-b7c7-328cabf42d81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:24:01.676568 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8243e982-4dc9-4431-b7c7-328cabf42d81 admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:24:01.676700 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8243e982-4dc9-4431-b7c7-328cabf42d81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:24:01.676977 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8243e982-4dc9-4431-b7c7-328cabf42d81 admin admin] PATH_INFO: `/v3/users` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:24:01.678965 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8243e982-4dc9-4431-b7c7-328cabf42d81 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:24:01.680933 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8243e982-4dc9-4431-b7c7-328cabf42d81 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:24:01.821433 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-eee017a4-c069-41e3-91b6-fb8718b6d532 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f96a4371b173477cb1b899490873dba0', 'request_id': 'req-eee017a4-c069-41e3-91b6-fb8718b6d532'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:24:01.826801 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-eee017a4-c069-41e3-91b6-fb8718b6d532 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f96a4371b173477cb1b899490873dba0', 'request_id': 'req-eee017a4-c069-41e3-91b6-fb8718b6d532'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:24:01.842599 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1282/2541] 10.222.0.23 () {60 vars in 1143 bytes} [Sat Jun 18 05:24:01 2022] POST /identity/v3/users => generated 292 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:24:01.843818 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8243e982-4dc9-4431-b7c7-328cabf42d81 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:24:01.854412 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8243e982-4dc9-4431-b7c7-328cabf42d81 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:24:01.907092 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-eee017a4-c069-41e3-91b6-fb8718b6d532 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f96a4371b173477cb1b899490873dba0', 'request_id': 'req-eee017a4-c069-41e3-91b6-fb8718b6d532'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:24:01.910988 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e6a378-be96-4cdf-bd97-846616ae4171 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:24:01.912864 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e6a378-be96-4cdf-bd97-846616ae4171 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:24:01.913203 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e6a378-be96-4cdf-bd97-846616ae4171 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:24:01.913542 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e6a378-be96-4cdf-bd97-846616ae4171 admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:24:01.917274 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e6a378-be96-4cdf-bd97-846616ae4171 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:24:01.917274 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e6a378-be96-4cdf-bd97-846616ae4171 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:24:01.919950 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-eee017a4-c069-41e3-91b6-fb8718b6d532 admin admin] The token cache is being invalidate because project f96a4371b173477cb1b899490873dba0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80468) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 18 05:24:01.920243 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-eee017a4-c069-41e3-91b6-fb8718b6d532 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:24:01.920894 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-eee017a4-c069-41e3-91b6-fb8718b6d532 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:24:01.923012 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1260/2542] 10.222.0.23 () {58 vars in 1232 bytes} [Sat Jun 18 05:24:01 2022] DELETE /identity/v3/projects/f96a4371b173477cb1b899490873dba0 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 18 05:24:01.929791 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1283/2543] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:24:01 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:24:01.929971 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3057ea54-5c0b-4112-a65a-cadf67cfaec2 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:24:01.929971 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3057ea54-5c0b-4112-a65a-cadf67cfaec2 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:24:01.929971 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3057ea54-5c0b-4112-a65a-cadf67cfaec2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:24:01.930716 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e6a378-be96-4cdf-bd97-846616ae4171 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:24:01.932332 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-10e6a378-be96-4cdf-bd97-846616ae4171 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:24:01.945933 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e6a378-be96-4cdf-bd97-846616ae4171 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:24:01.950821 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c989a2e9-920e-45ae-8b87-252865250da2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:24:01.951779 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c989a2e9-920e-45ae-8b87-252865250da2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:24:01.952066 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c989a2e9-920e-45ae-8b87-252865250da2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:24:01.952367 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c989a2e9-920e-45ae-8b87-252865250da2 admin admin] PATH_INFO: `/v3/projects/fa64432c05014fb5a8ab6a33254079d5/users/8c2a75fb3e2f49fd9b5fdc2b8d0bc1e6/roles/21c65f2c51bd4dd283d13507d2714467` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:24:01.955725 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-3057ea54-5c0b-4112-a65a-cadf67cfaec2 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:24:01.956936 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-3057ea54-5c0b-4112-a65a-cadf67cfaec2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:24:01.972000 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1261/2544] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:24:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:24:01.972164 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c989a2e9-920e-45ae-8b87-252865250da2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa64432c05014fb5a8ab6a33254079d5, user_id=8c2a75fb3e2f49fd9b5fdc2b8d0bc1e6, role_id=21c65f2c51bd4dd283d13507d2714467)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:24:01.972767 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c989a2e9-920e-45ae-8b87-252865250da2 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:24:01.973564 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4021f6-d088-4c07-92e8-ec1948b6e2b0 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:24:01.973937 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4021f6-d088-4c07-92e8-ec1948b6e2b0 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:24:01.975864 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4021f6-d088-4c07-92e8-ec1948b6e2b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:24:01.994206 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-2b4021f6-d088-4c07-92e8-ec1948b6e2b0 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:24:01.999695 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-2b4021f6-d088-4c07-92e8-ec1948b6e2b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:24:02.003537 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1262/2545] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:24:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:24:02.008117 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4021f6-d088-4c07-92e8-ec1948b6e2b0 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:24:02.017210 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4021f6-d088-4c07-92e8-ec1948b6e2b0 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:24:02.022694 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfacb7a8-a5a2-49ce-8bfd-0d16d9468e0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:24:02.028339 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfacb7a8-a5a2-49ce-8bfd-0d16d9468e0c admin admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:24:02.028339 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfacb7a8-a5a2-49ce-8bfd-0d16d9468e0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:24:02.028339 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfacb7a8-a5a2-49ce-8bfd-0d16d9468e0c admin admin] PATH_INFO: `/v3/domains` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:24:02.028339 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfacb7a8-a5a2-49ce-8bfd-0d16d9468e0c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 05:24:02.028339 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfacb7a8-a5a2-49ce-8bfd-0d16d9468e0c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:24:02.028339 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfacb7a8-a5a2-49ce-8bfd-0d16d9468e0c admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:24:02.039599 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1263/2546] 10.222.0.23 () {58 vars in 1152 bytes} [Sat Jun 18 05:24:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 18 05:24:02.043799 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfacb7a8-a5a2-49ce-8bfd-0d16d9468e0c admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:24:02.052958 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1284/2547] 10.222.0.23 () {60 vars in 1482 bytes} [Sat Jun 18 05:24:01 2022] PUT /identity/v3/projects/fa64432c05014fb5a8ab6a33254079d5/users/8c2a75fb3e2f49fd9b5fdc2b8d0bc1e6/roles/21c65f2c51bd4dd283d13507d2714467 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 18 05:24:02.054829 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfacb7a8-a5a2-49ce-8bfd-0d16d9468e0c admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:24:02.056678 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c989a2e9-920e-45ae-8b87-252865250da2 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:24:02.116361 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110edc10-38dc-4fec-90ef-f310fa4b02f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:24:02.117083 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110edc10-38dc-4fec-90ef-f310fa4b02f8 admin admin] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:24:02.117375 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110edc10-38dc-4fec-90ef-f310fa4b02f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:24:02.117691 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110edc10-38dc-4fec-90ef-f310fa4b02f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:24:02.118575 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110edc10-38dc-4fec-90ef-f310fa4b02f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:24:02.121562 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110edc10-38dc-4fec-90ef-f310fa4b02f8 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:24:02.123998 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c989a2e9-920e-45ae-8b87-252865250da2 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:24:02.124962 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:24:02.129034 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e56d17-7dd6-4859-8d17-c8fa8f79e3d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:24:02.131483 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e56d17-7dd6-4859-8d17-c8fa8f79e3d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:24:02.131483 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e56d17-7dd6-4859-8d17-c8fa8f79e3d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:24:02.131483 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e56d17-7dd6-4859-8d17-c8fa8f79e3d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:24:02.131917 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e56d17-7dd6-4859-8d17-c8fa8f79e3d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:24:02.133145 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e56d17-7dd6-4859-8d17-c8fa8f79e3d8 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:24:02.147112 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1285/2548] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:24:02 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 18 05:24:02.148381 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e56d17-7dd6-4859-8d17-c8fa8f79e3d8 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:24:02.156211 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:24:02.168792 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e56d17-7dd6-4859-8d17-c8fa8f79e3d8 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:24:02.173960 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc21a64-750a-43bf-9628-400ae20a79ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:24:02.174847 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc21a64-750a-43bf-9628-400ae20a79ea admin admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:24:02.175132 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc21a64-750a-43bf-9628-400ae20a79ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:24:02.175562 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc21a64-750a-43bf-9628-400ae20a79ea admin admin] PATH_INFO: `/v3/projects/fa64432c05014fb5a8ab6a33254079d5/users/8c2a75fb3e2f49fd9b5fdc2b8d0bc1e6/roles/21c65f2c51bd4dd283d13507d2714467` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:24:02.178029 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc21a64-750a-43bf-9628-400ae20a79ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa64432c05014fb5a8ab6a33254079d5, user_id=8c2a75fb3e2f49fd9b5fdc2b8d0bc1e6, role_id=21c65f2c51bd4dd283d13507d2714467)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:24:02.180728 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc21a64-750a-43bf-9628-400ae20a79ea admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:24:02.192063 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1286/2549] 10.222.0.23 () {60 vars in 1482 bytes} [Sat Jun 18 05:24:02 2022] PUT /identity/v3/projects/fa64432c05014fb5a8ab6a33254079d5/users/8c2a75fb3e2f49fd9b5fdc2b8d0bc1e6/roles/21c65f2c51bd4dd283d13507d2714467 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 18 05:24:02.202877 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdba29a-f197-400f-9377-491ab4bf0999 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:24:02.203398 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdba29a-f197-400f-9377-491ab4bf0999 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:24:02.203682 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdba29a-f197-400f-9377-491ab4bf0999 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:24:02.214676 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1264/2550] 10.222.0.23 () {60 vars in 1152 bytes} [Sat Jun 18 05:24:02 2022] POST /identity/v3/projects => generated 359 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 18 05:24:02.220375 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110edc10-38dc-4fec-90ef-f310fa4b02f8 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:24:02.243320 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-9fdba29a-f197-400f-9377-491ab4bf0999 None None] MFA Rules not processed for user `8c2a75fb3e2f49fd9b5fdc2b8d0bc1e6`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:24:02.244609 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-9fdba29a-f197-400f-9377-491ab4bf0999 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:24:02.289814 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110edc10-38dc-4fec-90ef-f310fa4b02f8 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:24:02.294494 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae2e24f-b380-495e-9cb8-df306ad5315b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:24:02.295899 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae2e24f-b380-495e-9cb8-df306ad5315b admin admin] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:24:02.296118 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae2e24f-b380-495e-9cb8-df306ad5315b admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:24:02.296401 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae2e24f-b380-495e-9cb8-df306ad5315b admin admin] PATH_INFO: `/v3/users` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:24:02.297085 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae2e24f-b380-495e-9cb8-df306ad5315b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:24:02.299638 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae2e24f-b380-495e-9cb8-df306ad5315b admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:24:02.315200 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1287/2551] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:24:02 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 18 05:24:02.327152 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdba29a-f197-400f-9377-491ab4bf0999 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:24:02.329309 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-9fdba29a-f197-400f-9377-491ab4bf0999 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:24:02.387347 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1265/2552] 10.222.0.23 () {60 vars in 1143 bytes} [Sat Jun 18 05:24:02 2022] POST /identity/v3/users => generated 280 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:24:02.387547 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdba29a-f197-400f-9377-491ab4bf0999 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:24:02.391805 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae2e24f-b380-495e-9cb8-df306ad5315b admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:24:02.394092 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138bc73a-1c41-4119-b43a-5bcc22de0da9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9de0428abc8400aa338aba4f7c34c94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:24:02.394809 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138bc73a-1c41-4119-b43a-5bcc22de0da9 service cinder] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:24:02.395093 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138bc73a-1c41-4119-b43a-5bcc22de0da9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:24:02.395369 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138bc73a-1c41-4119-b43a-5bcc22de0da9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:24:02.403598 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae2e24f-b380-495e-9cb8-df306ad5315b admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:24:02.405802 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138bc73a-1c41-4119-b43a-5bcc22de0da9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:24:02.407250 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138bc73a-1c41-4119-b43a-5bcc22de0da9 service cinder] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:24:02.408241 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3b4b4f-2a2e-4f30-a0aa-7b5cdd812532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:24:02.408888 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3b4b4f-2a2e-4f30-a0aa-7b5cdd812532 admin admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:24:02.409172 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3b4b4f-2a2e-4f30-a0aa-7b5cdd812532 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:24:02.409490 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3b4b4f-2a2e-4f30-a0aa-7b5cdd812532 admin admin] PATH_INFO: `/v3/roles` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:24:02.410311 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3b4b4f-2a2e-4f30-a0aa-7b5cdd812532 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:24:02.412185 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3b4b4f-2a2e-4f30-a0aa-7b5cdd812532 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:24:02.417322 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1288/2553] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:24:02 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 18 05:24:02.422671 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1266/2554] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:24:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:24:02.426856 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138bc73a-1c41-4119-b43a-5bcc22de0da9 service cinder] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:24:02.437457 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138bc73a-1c41-4119-b43a-5bcc22de0da9 service cinder] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:24:02.443563 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f999e2-e1f1-4428-9d46-c6d54b239629 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:24:02.443563 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f999e2-e1f1-4428-9d46-c6d54b239629 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:24:02.443563 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f999e2-e1f1-4428-9d46-c6d54b239629 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:24:02.444015 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f999e2-e1f1-4428-9d46-c6d54b239629 admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:24:02.444475 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f999e2-e1f1-4428-9d46-c6d54b239629 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:24:02.446128 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f999e2-e1f1-4428-9d46-c6d54b239629 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:24:02.455902 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1289/2555] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:24:02 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:24:02.460133 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3b4b4f-2a2e-4f30-a0aa-7b5cdd812532 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:24:02.470174 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3b4b4f-2a2e-4f30-a0aa-7b5cdd812532 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:24:02.475612 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac82767f-7b5a-43e9-97c3-0edd62788758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:24:02.476525 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac82767f-7b5a-43e9-97c3-0edd62788758 admin admin] REQUEST_METHOD: `PUT` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:24:02.476806 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac82767f-7b5a-43e9-97c3-0edd62788758 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:24:02.477082 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac82767f-7b5a-43e9-97c3-0edd62788758 admin admin] PATH_INFO: `/v3/projects/208c6fa2d4724d7ba8136da69fd8755e/users/b3d36998e9f4414e9386abc76b4c43be/roles/cbf5c4c26898413d93e73b46b41046a3` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:24:02.492370 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac82767f-7b5a-43e9-97c3-0edd62788758 admin admin] RBAC: Authorizing `identity:create_grant(project_id=208c6fa2d4724d7ba8136da69fd8755e, user_id=b3d36998e9f4414e9386abc76b4c43be, role_id=cbf5c4c26898413d93e73b46b41046a3)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:24:02.495072 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac82767f-7b5a-43e9-97c3-0edd62788758 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:24:02.571809 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1267/2556] 10.222.0.23 () {60 vars in 1482 bytes} [Sat Jun 18 05:24:02 2022] PUT /identity/v3/projects/208c6fa2d4724d7ba8136da69fd8755e/users/b3d36998e9f4414e9386abc76b4c43be/roles/cbf5c4c26898413d93e73b46b41046a3 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:24:02.581879 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a883a6fa-0e43-4762-994d-246dd6672274 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:24:02.582168 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a883a6fa-0e43-4762-994d-246dd6672274 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:24:02.582449 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a883a6fa-0e43-4762-994d-246dd6672274 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:24:02.605827 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:24:02.613723 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac82767f-7b5a-43e9-97c3-0edd62788758 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:24:02.616225 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-a883a6fa-0e43-4762-994d-246dd6672274 None None] MFA Rules not processed for user `b3d36998e9f4414e9386abc76b4c43be`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:24:02.617491 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-a883a6fa-0e43-4762-994d-246dd6672274 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:24:02.670398 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:24:02.674345 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac82767f-7b5a-43e9-97c3-0edd62788758 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:24:02.678297 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecede8a0-a78c-4552-a8ae-4b2e2cf7b738 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9de0428abc8400aa338aba4f7c34c94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:24:02.678998 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecede8a0-a78c-4552-a8ae-4b2e2cf7b738 service cinder] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:24:02.679282 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecede8a0-a78c-4552-a8ae-4b2e2cf7b738 service cinder] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:24:02.679554 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecede8a0-a78c-4552-a8ae-4b2e2cf7b738 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:24:02.681518 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-ecede8a0-a78c-4552-a8ae-4b2e2cf7b738 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:24:02.685903 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1290/2557] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:24:02 2022] POST /identity/v3/auth/tokens => generated 3226 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:24:02.723815 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a883a6fa-0e43-4762-994d-246dd6672274 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:24:02.725310 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-a883a6fa-0e43-4762-994d-246dd6672274 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:24:02.737192 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a883a6fa-0e43-4762-994d-246dd6672274 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:24:02.741831 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdb5527-3d70-4e09-a408-e24fd41235e4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9de0428abc8400aa338aba4f7c34c94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:24:02.742476 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdb5527-3d70-4e09-a408-e24fd41235e4 service cinder] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:24:02.742759 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdb5527-3d70-4e09-a408-e24fd41235e4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:24:02.743030 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdb5527-3d70-4e09-a408-e24fd41235e4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:24:02.749958 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecede8a0-a78c-4552-a8ae-4b2e2cf7b738 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:24:02.751398 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecede8a0-a78c-4552-a8ae-4b2e2cf7b738 service cinder] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:24:02.752099 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdb5527-3d70-4e09-a408-e24fd41235e4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:24:02.753601 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdb5527-3d70-4e09-a408-e24fd41235e4 service cinder] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:24:02.760836 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1268/2558] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:24:02 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:24:02.763928 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1291/2559] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:24:02 2022] GET /identity/v3/auth/tokens => generated 3226 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 18 05:24:02.779653 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecede8a0-a78c-4552-a8ae-4b2e2cf7b738 service cinder] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:24:02.843966 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecede8a0-a78c-4552-a8ae-4b2e2cf7b738 service cinder] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:24:02.848597 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da01dcbb-ebc0-4870-9149-c9be75b22f36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:24:02.866786 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da01dcbb-ebc0-4870-9149-c9be75b22f36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:24:02.866786 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da01dcbb-ebc0-4870-9149-c9be75b22f36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:24:02.866786 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da01dcbb-ebc0-4870-9149-c9be75b22f36 admin admin] PATH_INFO: `/v3/users/b3d36998e9f4414e9386abc76b4c43be` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:24:02.866786 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da01dcbb-ebc0-4870-9149-c9be75b22f36 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3d36998e9f4414e9386abc76b4c43be)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:24:02.866786 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da01dcbb-ebc0-4870-9149-c9be75b22f36 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:24:03.054381 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-da01dcbb-ebc0-4870-9149-c9be75b22f36 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3d36998e9f4414e9386abc76b4c43be'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:24:03.055492 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-da01dcbb-ebc0-4870-9149-c9be75b22f36 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3d36998e9f4414e9386abc76b4c43be'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:24:03.103095 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-da01dcbb-ebc0-4870-9149-c9be75b22f36 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3d36998e9f4414e9386abc76b4c43be'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:24:03.118121 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-da01dcbb-ebc0-4870-9149-c9be75b22f36 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3d36998e9f4414e9386abc76b4c43be'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:24:03.125994 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-da01dcbb-ebc0-4870-9149-c9be75b22f36 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3d36998e9f4414e9386abc76b4c43be'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:24:03.135522 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1269/2560] 10.222.0.23 () {58 vars in 1223 bytes} [Sat Jun 18 05:24:02 2022] DELETE /identity/v3/users/b3d36998e9f4414e9386abc76b4c43be => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:24:03.137722 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b55b89-9df7-4ea3-8792-271f62898e68 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:24:03.138227 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b55b89-9df7-4ea3-8792-271f62898e68 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:24:03.138320 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b55b89-9df7-4ea3-8792-271f62898e68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:24:03.170635 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-54b55b89-9df7-4ea3-8792-271f62898e68 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:24:03.171977 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-54b55b89-9df7-4ea3-8792-271f62898e68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:24:03.240112 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1292/2561] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:24:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 18 05:24:03.250436 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da01dcbb-ebc0-4870-9149-c9be75b22f36 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:24:03.253533 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-da01dcbb-ebc0-4870-9149-c9be75b22f36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:24:03.290593 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:24:03.327143 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da01dcbb-ebc0-4870-9149-c9be75b22f36 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:24:03.332499 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6a4fa6-48f3-4e45-8765-23568f04f6fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c4d7cde8f4ad9aa479bb09e325db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:24:03.333234 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6a4fa6-48f3-4e45-8765-23568f04f6fe service neutron] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:24:03.333813 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6a4fa6-48f3-4e45-8765-23568f04f6fe service neutron] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:24:03.334005 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6a4fa6-48f3-4e45-8765-23568f04f6fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:24:03.344474 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6a4fa6-48f3-4e45-8765-23568f04f6fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:24:03.345968 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6a4fa6-48f3-4e45-8765-23568f04f6fe service neutron] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:24:03.372742 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1270/2562] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:24:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:24:03.914729 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b55b89-9df7-4ea3-8792-271f62898e68 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:24:03.916297 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-54b55b89-9df7-4ea3-8792-271f62898e68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:24:03.928537 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b55b89-9df7-4ea3-8792-271f62898e68 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:24:03.940016 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136a7911-9a35-426f-9290-bdbb10ca46b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:24:03.941029 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136a7911-9a35-426f-9290-bdbb10ca46b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:24:03.941412 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136a7911-9a35-426f-9290-bdbb10ca46b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:24:03.941773 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136a7911-9a35-426f-9290-bdbb10ca46b3 admin admin] PATH_INFO: `/v3/projects/208c6fa2d4724d7ba8136da69fd8755e` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:24:03.943908 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136a7911-9a35-426f-9290-bdbb10ca46b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=208c6fa2d4724d7ba8136da69fd8755e)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:24:03.946165 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136a7911-9a35-426f-9290-bdbb10ca46b3 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:24:04.049181 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-136a7911-9a35-426f-9290-bdbb10ca46b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '208c6fa2d4724d7ba8136da69fd8755e', 'request_id': 'req-136a7911-9a35-426f-9290-bdbb10ca46b3'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:24:04.055844 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-136a7911-9a35-426f-9290-bdbb10ca46b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '208c6fa2d4724d7ba8136da69fd8755e', 'request_id': 'req-136a7911-9a35-426f-9290-bdbb10ca46b3'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:24:04.062251 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-136a7911-9a35-426f-9290-bdbb10ca46b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '208c6fa2d4724d7ba8136da69fd8755e', 'request_id': 'req-136a7911-9a35-426f-9290-bdbb10ca46b3'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:24:04.120261 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-136a7911-9a35-426f-9290-bdbb10ca46b3 admin admin] The token cache is being invalidate because project 208c6fa2d4724d7ba8136da69fd8755e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80469) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 18 05:24:04.120550 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-136a7911-9a35-426f-9290-bdbb10ca46b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:24:04.121239 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-136a7911-9a35-426f-9290-bdbb10ca46b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:24:04.122794 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1293/2563] 10.222.0.23 () {58 vars in 1232 bytes} [Sat Jun 18 05:24:03 2022] DELETE /identity/v3/projects/208c6fa2d4724d7ba8136da69fd8755e => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:24:04.127824 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916f84c3-5bc3-4851-a757-5a4334896eff None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:24:04.128296 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916f84c3-5bc3-4851-a757-5a4334896eff None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:24:04.128536 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916f84c3-5bc3-4851-a757-5a4334896eff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:24:04.147895 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-916f84c3-5bc3-4851-a757-5a4334896eff None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:24:04.149156 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-916f84c3-5bc3-4851-a757-5a4334896eff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:24:04.216167 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1271/2564] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:24:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 18 05:24:04.226091 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a18b8bb-ff33-40a5-8262-e3bc09ed314b None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:24:04.226616 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a18b8bb-ff33-40a5-8262-e3bc09ed314b None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:24:04.226712 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a18b8bb-ff33-40a5-8262-e3bc09ed314b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:24:04.247424 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-8a18b8bb-ff33-40a5-8262-e3bc09ed314b None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:24:04.248447 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-8a18b8bb-ff33-40a5-8262-e3bc09ed314b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:24:04.268394 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916f84c3-5bc3-4851-a757-5a4334896eff None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:24:04.268706 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1294/2565] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:24:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 18 05:24:04.271925 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916f84c3-5bc3-4851-a757-5a4334896eff None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:24:04.276713 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef01617-5a91-48f5-be56-bc6df85ed3dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:24:04.277481 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef01617-5a91-48f5-be56-bc6df85ed3dc admin admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:24:04.277766 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef01617-5a91-48f5-be56-bc6df85ed3dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:24:04.278046 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef01617-5a91-48f5-be56-bc6df85ed3dc admin admin] PATH_INFO: `/v3/domains` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:24:04.278766 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef01617-5a91-48f5-be56-bc6df85ed3dc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 05:24:04.279076 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef01617-5a91-48f5-be56-bc6df85ed3dc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:24:04.280928 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef01617-5a91-48f5-be56-bc6df85ed3dc admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:24:04.293142 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1272/2566] 10.222.0.23 () {58 vars in 1152 bytes} [Sat Jun 18 05:24:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 05:24:04.297497 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a18b8bb-ff33-40a5-8262-e3bc09ed314b None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:24:04.303333 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:24:04.308997 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a18b8bb-ff33-40a5-8262-e3bc09ed314b None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:24:04.320010 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc8be16-7278-4230-a09d-6d7799d68f45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:24:04.320010 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc8be16-7278-4230-a09d-6d7799d68f45 admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:24:04.320010 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc8be16-7278-4230-a09d-6d7799d68f45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:24:04.320010 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc8be16-7278-4230-a09d-6d7799d68f45 admin admin] PATH_INFO: `/v3/projects` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:24:04.320402 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc8be16-7278-4230-a09d-6d7799d68f45 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:24:04.320402 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc8be16-7278-4230-a09d-6d7799d68f45 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:24:04.402207 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1295/2567] 10.222.0.23 () {60 vars in 1152 bytes} [Sat Jun 18 05:24:04 2022] POST /identity/v3/projects => generated 383 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:24:04.403587 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef01617-5a91-48f5-be56-bc6df85ed3dc admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:24:04.419726 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef01617-5a91-48f5-be56-bc6df85ed3dc admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:24:04.507577 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad870a1b-379e-48ed-bae4-89cbc83a8528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:24:04.508557 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad870a1b-379e-48ed-bae4-89cbc83a8528 admin admin] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:24:04.508943 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad870a1b-379e-48ed-bae4-89cbc83a8528 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:24:04.509323 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad870a1b-379e-48ed-bae4-89cbc83a8528 admin admin] PATH_INFO: `/v3/users` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:24:04.510147 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad870a1b-379e-48ed-bae4-89cbc83a8528 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:24:04.512252 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad870a1b-379e-48ed-bae4-89cbc83a8528 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:24:04.609456 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1273/2568] 10.222.0.23 () {60 vars in 1143 bytes} [Sat Jun 18 05:24:04 2022] POST /identity/v3/users => generated 292 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 18 05:24:04.611329 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc8be16-7278-4230-a09d-6d7799d68f45 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:24:04.622827 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc8be16-7278-4230-a09d-6d7799d68f45 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:24:04.628683 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b57009-4645-45d0-b9e0-687a9a178080 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:24:04.629778 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b57009-4645-45d0-b9e0-687a9a178080 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:24:04.630850 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b57009-4645-45d0-b9e0-687a9a178080 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:24:04.631278 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b57009-4645-45d0-b9e0-687a9a178080 admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:24:04.632588 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b57009-4645-45d0-b9e0-687a9a178080 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:24:04.635395 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b57009-4645-45d0-b9e0-687a9a178080 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:24:04.650130 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1296/2569] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:24:04 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:24:04.654194 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad870a1b-379e-48ed-bae4-89cbc83a8528 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:24:04.666021 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad870a1b-379e-48ed-bae4-89cbc83a8528 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:24:04.671781 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c2d31a-5b78-40da-a2ea-a0e4f71da487 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:24:04.672635 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c2d31a-5b78-40da-a2ea-a0e4f71da487 admin admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:24:04.672811 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c2d31a-5b78-40da-a2ea-a0e4f71da487 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:24:04.673099 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c2d31a-5b78-40da-a2ea-a0e4f71da487 admin admin] PATH_INFO: `/v3/roles` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:24:04.673867 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c2d31a-5b78-40da-a2ea-a0e4f71da487 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:24:04.675948 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c2d31a-5b78-40da-a2ea-a0e4f71da487 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:24:04.688417 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1274/2570] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:24:04 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:24:04.692596 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b57009-4645-45d0-b9e0-687a9a178080 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:24:04.703180 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b57009-4645-45d0-b9e0-687a9a178080 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:24:04.707899 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087b7341-b7b5-4234-999e-d9df58fa019e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:24:04.708824 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087b7341-b7b5-4234-999e-d9df58fa019e admin admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:24:04.709106 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087b7341-b7b5-4234-999e-d9df58fa019e admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:24:04.709397 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087b7341-b7b5-4234-999e-d9df58fa019e admin admin] PATH_INFO: `/v3/projects/d8863ced0efc4b95806761cc2d8a6f78/users/f54c1aed6a784e40952d38784bf4ed6b/roles/cbf5c4c26898413d93e73b46b41046a3` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:24:04.725650 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087b7341-b7b5-4234-999e-d9df58fa019e admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8863ced0efc4b95806761cc2d8a6f78, user_id=f54c1aed6a784e40952d38784bf4ed6b, role_id=cbf5c4c26898413d93e73b46b41046a3)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:24:04.730685 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087b7341-b7b5-4234-999e-d9df58fa019e admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:24:04.839769 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1297/2571] 10.222.0.23 () {60 vars in 1482 bytes} [Sat Jun 18 05:24:04 2022] PUT /identity/v3/projects/d8863ced0efc4b95806761cc2d8a6f78/users/f54c1aed6a784e40952d38784bf4ed6b/roles/cbf5c4c26898413d93e73b46b41046a3 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:24:04.840120 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92a87cc-32e0-4f48-8888-06a7ddcd3427 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:24:04.840120 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92a87cc-32e0-4f48-8888-06a7ddcd3427 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:24:04.840120 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92a87cc-32e0-4f48-8888-06a7ddcd3427 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:24:04.860539 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-a92a87cc-32e0-4f48-8888-06a7ddcd3427 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:24:04.862443 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-a92a87cc-32e0-4f48-8888-06a7ddcd3427 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:24:04.895337 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:24:04.940590 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1275/2572] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:24:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:24:04.954738 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087b7341-b7b5-4234-999e-d9df58fa019e admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:24:04.954738 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-087b7341-b7b5-4234-999e-d9df58fa019e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:24:05.025917 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087b7341-b7b5-4234-999e-d9df58fa019e admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:24:05.029765 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4348c824-06fb-4d02-b358-6a64dac74278 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c4d7cde8f4ad9aa479bb09e325db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:24:05.030856 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4348c824-06fb-4d02-b358-6a64dac74278 service neutron] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:24:05.035342 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4348c824-06fb-4d02-b358-6a64dac74278 service neutron] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:24:05.035342 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4348c824-06fb-4d02-b358-6a64dac74278 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:24:05.042264 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4348c824-06fb-4d02-b358-6a64dac74278 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:24:05.043991 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4348c824-06fb-4d02-b358-6a64dac74278 service neutron] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:24:05.055634 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1298/2573] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:24:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:24:06.703006 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3667bdc1-f809-4066-b3b3-426053c25076 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:24:06.703478 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3667bdc1-f809-4066-b3b3-426053c25076 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:24:06.703596 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3667bdc1-f809-4066-b3b3-426053c25076 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:24:06.730640 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-3667bdc1-f809-4066-b3b3-426053c25076 None None] MFA Rules not processed for user `f54c1aed6a784e40952d38784bf4ed6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:24:06.731863 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-3667bdc1-f809-4066-b3b3-426053c25076 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:24:06.799175 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1276/2574] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:24:06 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 18 05:24:06.810654 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4348c824-06fb-4d02-b358-6a64dac74278 service neutron] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:24:06.812082 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-4348c824-06fb-4d02-b358-6a64dac74278 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:24:06.870487 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4348c824-06fb-4d02-b358-6a64dac74278 service neutron] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:24:06.874101 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e62ddad-4a78-414c-ab32-850e73dbcf1c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9de0428abc8400aa338aba4f7c34c94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:24:06.874738 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e62ddad-4a78-414c-ab32-850e73dbcf1c service cinder] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:24:06.875017 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e62ddad-4a78-414c-ab32-850e73dbcf1c service cinder] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:24:06.875291 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e62ddad-4a78-414c-ab32-850e73dbcf1c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:24:06.884220 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e62ddad-4a78-414c-ab32-850e73dbcf1c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:24:06.885694 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e62ddad-4a78-414c-ab32-850e73dbcf1c service cinder] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:24:06.894642 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1299/2575] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:24:06 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:24:10.701940 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00c85c4-89b3-4df6-9f36-4d73c25e59f2 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:24:10.702466 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00c85c4-89b3-4df6-9f36-4d73c25e59f2 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:24:10.702563 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00c85c4-89b3-4df6-9f36-4d73c25e59f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:24:10.719999 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-b00c85c4-89b3-4df6-9f36-4d73c25e59f2 None None] MFA Rules not processed for user `f54c1aed6a784e40952d38784bf4ed6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:24:10.721199 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-b00c85c4-89b3-4df6-9f36-4d73c25e59f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:24:10.729151 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1277/2576] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:24:10 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:24:16.890634 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=75,keystone:INSERT=2 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:24:20.714651 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=48 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:24:22.202570 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e62ddad-4a78-414c-ab32-850e73dbcf1c service cinder] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:24:22.204018 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-6e62ddad-4a78-414c-ab32-850e73dbcf1c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:24:22.216643 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e62ddad-4a78-414c-ab32-850e73dbcf1c service cinder] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:24:22.220342 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd0ccf4-a595-4de6-8f15-f05adc748d0c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c4d7cde8f4ad9aa479bb09e325db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:24:22.224220 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd0ccf4-a595-4de6-8f15-f05adc748d0c service neutron] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:24:22.224220 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd0ccf4-a595-4de6-8f15-f05adc748d0c service neutron] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:24:22.224220 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd0ccf4-a595-4de6-8f15-f05adc748d0c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:24:22.224220 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-5cd0ccf4-a595-4de6-8f15-f05adc748d0c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:24:22.285588 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd0ccf4-a595-4de6-8f15-f05adc748d0c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:24:22.287753 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd0ccf4-a595-4de6-8f15-f05adc748d0c service neutron] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:24:22.297448 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1300/2577] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:24:22 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:24:29.694331 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00c85c4-89b3-4df6-9f36-4d73c25e59f2 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:24:29.707207 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00c85c4-89b3-4df6-9f36-4d73c25e59f2 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:24:29.712178 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acbea1c-bfd1-442d-b086-b905eb072125 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c4d7cde8f4ad9aa479bb09e325db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:24:29.713132 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acbea1c-bfd1-442d-b086-b905eb072125 service neutron] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:24:29.713132 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acbea1c-bfd1-442d-b086-b905eb072125 service neutron] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:24:29.713439 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acbea1c-bfd1-442d-b086-b905eb072125 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:24:29.715207 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-3acbea1c-bfd1-442d-b086-b905eb072125 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:24:29.727805 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acbea1c-bfd1-442d-b086-b905eb072125 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:24:29.729281 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acbea1c-bfd1-442d-b086-b905eb072125 service neutron] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:24:29.739145 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1278/2578] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:24:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:24:32.293505 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=28 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:24:37.448913 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd0ccf4-a595-4de6-8f15-f05adc748d0c service neutron] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:24:37.451506 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-5cd0ccf4-a595-4de6-8f15-f05adc748d0c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:24:37.466365 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd0ccf4-a595-4de6-8f15-f05adc748d0c service neutron] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:24:37.471123 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7c0a7c-5305-41b0-9365-bf85e40301be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:24:37.472376 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7c0a7c-5305-41b0-9365-bf85e40301be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:24:37.472376 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7c0a7c-5305-41b0-9365-bf85e40301be admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:24:37.472587 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7c0a7c-5305-41b0-9365-bf85e40301be admin admin] PATH_INFO: `/v3/users/55bc735165cd4029bd5d581fd0187f9b` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:24:37.473730 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7c0a7c-5305-41b0-9365-bf85e40301be admin admin] RBAC: Authorizing `identity:delete_user(user_id=55bc735165cd4029bd5d581fd0187f9b)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:24:37.475684 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7c0a7c-5305-41b0-9365-bf85e40301be admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:24:37.744269 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-7c7c0a7c-5305-41b0-9365-bf85e40301be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55bc735165cd4029bd5d581fd0187f9b'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:24:37.756163 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-7c7c0a7c-5305-41b0-9365-bf85e40301be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55bc735165cd4029bd5d581fd0187f9b'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:24:37.756849 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-7c7c0a7c-5305-41b0-9365-bf85e40301be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55bc735165cd4029bd5d581fd0187f9b'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:24:37.815637 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-7c7c0a7c-5305-41b0-9365-bf85e40301be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55bc735165cd4029bd5d581fd0187f9b'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:24:37.824090 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-7c7c0a7c-5305-41b0-9365-bf85e40301be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55bc735165cd4029bd5d581fd0187f9b'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:24:37.830142 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1301/2579] 10.222.0.23 () {58 vars in 1223 bytes} [Sat Jun 18 05:24:37 2022] DELETE /identity/v3/users/55bc735165cd4029bd5d581fd0187f9b => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:24:38.086918 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acbea1c-bfd1-442d-b086-b905eb072125 service neutron] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:24:38.099288 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acbea1c-bfd1-442d-b086-b905eb072125 service neutron] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:24:38.164448 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd626920-74a7-4990-b3c4-5918d422a030 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:24:38.167005 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd626920-74a7-4990-b3c4-5918d422a030 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:24:38.167005 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd626920-74a7-4990-b3c4-5918d422a030 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:24:38.167005 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd626920-74a7-4990-b3c4-5918d422a030 admin admin] PATH_INFO: `/v3/projects/65a7c712ebe34d53b4acd7e8473c6d95` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:24:38.167798 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd626920-74a7-4990-b3c4-5918d422a030 admin admin] RBAC: Authorizing `identity:delete_project(project_id=65a7c712ebe34d53b4acd7e8473c6d95)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:24:38.169329 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd626920-74a7-4990-b3c4-5918d422a030 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:24:38.308167 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-dd626920-74a7-4990-b3c4-5918d422a030 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '65a7c712ebe34d53b4acd7e8473c6d95', 'request_id': 'req-dd626920-74a7-4990-b3c4-5918d422a030'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:24:38.314163 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-dd626920-74a7-4990-b3c4-5918d422a030 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '65a7c712ebe34d53b4acd7e8473c6d95', 'request_id': 'req-dd626920-74a7-4990-b3c4-5918d422a030'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:24:38.381115 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-dd626920-74a7-4990-b3c4-5918d422a030 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '65a7c712ebe34d53b4acd7e8473c6d95', 'request_id': 'req-dd626920-74a7-4990-b3c4-5918d422a030'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:24:38.393597 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-dd626920-74a7-4990-b3c4-5918d422a030 admin admin] The token cache is being invalidate because project 65a7c712ebe34d53b4acd7e8473c6d95 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80468) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 18 05:24:38.393888 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-dd626920-74a7-4990-b3c4-5918d422a030 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:24:38.394559 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-dd626920-74a7-4990-b3c4-5918d422a030 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:24:38.396085 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1279/2580] 10.222.0.23 () {58 vars in 1232 bytes} [Sat Jun 18 05:24:38 2022] DELETE /identity/v3/projects/65a7c712ebe34d53b4acd7e8473c6d95 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:24:38.401399 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7c0a7c-5305-41b0-9365-bf85e40301be admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:24:38.402949 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-7c7c0a7c-5305-41b0-9365-bf85e40301be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:24:38.418003 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7c0a7c-5305-41b0-9365-bf85e40301be admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:24:38.478691 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8385eb94-1444-463e-95ca-b78cf2e5cb85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:24:38.479571 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8385eb94-1444-463e-95ca-b78cf2e5cb85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:24:38.479855 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8385eb94-1444-463e-95ca-b78cf2e5cb85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:24:38.480134 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8385eb94-1444-463e-95ca-b78cf2e5cb85 admin admin] PATH_INFO: `/v3/users/2d5c6fe0cb7e4da69cb69d8de910a82d` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:24:38.481337 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8385eb94-1444-463e-95ca-b78cf2e5cb85 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d5c6fe0cb7e4da69cb69d8de910a82d)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:24:38.483398 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8385eb94-1444-463e-95ca-b78cf2e5cb85 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:24:38.693796 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-8385eb94-1444-463e-95ca-b78cf2e5cb85 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d5c6fe0cb7e4da69cb69d8de910a82d'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:24:38.705612 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-8385eb94-1444-463e-95ca-b78cf2e5cb85 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d5c6fe0cb7e4da69cb69d8de910a82d'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:24:38.706276 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-8385eb94-1444-463e-95ca-b78cf2e5cb85 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d5c6fe0cb7e4da69cb69d8de910a82d'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:24:38.770654 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-8385eb94-1444-463e-95ca-b78cf2e5cb85 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d5c6fe0cb7e4da69cb69d8de910a82d'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:24:38.773770 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:24:38.779382 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-8385eb94-1444-463e-95ca-b78cf2e5cb85 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d5c6fe0cb7e4da69cb69d8de910a82d'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:24:38.785633 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1302/2581] 10.222.0.23 () {58 vars in 1223 bytes} [Sat Jun 18 05:24:38 2022] DELETE /identity/v3/users/2d5c6fe0cb7e4da69cb69d8de910a82d => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:24:39.080119 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd626920-74a7-4990-b3c4-5918d422a030 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:24:39.082121 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-dd626920-74a7-4990-b3c4-5918d422a030 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:24:39.097040 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd626920-74a7-4990-b3c4-5918d422a030 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:24:39.161293 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fb4782-a8e1-4959-a260-9db9703baad4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:24:39.162194 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fb4782-a8e1-4959-a260-9db9703baad4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:24:39.162477 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fb4782-a8e1-4959-a260-9db9703baad4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:24:39.162751 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fb4782-a8e1-4959-a260-9db9703baad4 admin admin] PATH_INFO: `/v3/projects/a42225556e894a4b83bf66b4a3cb2522` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:24:39.164127 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fb4782-a8e1-4959-a260-9db9703baad4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a42225556e894a4b83bf66b4a3cb2522)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:24:39.167346 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fb4782-a8e1-4959-a260-9db9703baad4 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:24:39.190225 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:24:39.302143 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-16fb4782-a8e1-4959-a260-9db9703baad4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a42225556e894a4b83bf66b4a3cb2522', 'request_id': 'req-16fb4782-a8e1-4959-a260-9db9703baad4'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:24:39.307762 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-16fb4782-a8e1-4959-a260-9db9703baad4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a42225556e894a4b83bf66b4a3cb2522', 'request_id': 'req-16fb4782-a8e1-4959-a260-9db9703baad4'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:24:39.393673 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-16fb4782-a8e1-4959-a260-9db9703baad4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a42225556e894a4b83bf66b4a3cb2522', 'request_id': 'req-16fb4782-a8e1-4959-a260-9db9703baad4'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:24:39.407453 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-16fb4782-a8e1-4959-a260-9db9703baad4 admin admin] The token cache is being invalidate because project a42225556e894a4b83bf66b4a3cb2522 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80468) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 18 05:24:39.407734 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-16fb4782-a8e1-4959-a260-9db9703baad4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:24:39.408410 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-16fb4782-a8e1-4959-a260-9db9703baad4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:24:39.410080 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1280/2582] 10.222.0.23 () {58 vars in 1232 bytes} [Sat Jun 18 05:24:39 2022] DELETE /identity/v3/projects/a42225556e894a4b83bf66b4a3cb2522 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 18 05:24:48.947759 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=4 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:24:49.467425 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:24:51.364012 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8385eb94-1444-463e-95ca-b78cf2e5cb85 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:24:51.365450 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-8385eb94-1444-463e-95ca-b78cf2e5cb85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:24:51.379716 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8385eb94-1444-463e-95ca-b78cf2e5cb85 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:24:51.438512 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae4d8c0-27fb-499d-bf20-05a829e41ed3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:24:51.439990 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae4d8c0-27fb-499d-bf20-05a829e41ed3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:24:51.440285 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae4d8c0-27fb-499d-bf20-05a829e41ed3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:24:51.440557 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae4d8c0-27fb-499d-bf20-05a829e41ed3 admin admin] PATH_INFO: `/v3/users/cc3cd4819cbf438fb2c5d4e9edb55360` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:24:51.441780 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae4d8c0-27fb-499d-bf20-05a829e41ed3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc3cd4819cbf438fb2c5d4e9edb55360)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:24:51.443736 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae4d8c0-27fb-499d-bf20-05a829e41ed3 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:24:51.618669 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-3ae4d8c0-27fb-499d-bf20-05a829e41ed3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc3cd4819cbf438fb2c5d4e9edb55360'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:24:51.632652 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-3ae4d8c0-27fb-499d-bf20-05a829e41ed3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc3cd4819cbf438fb2c5d4e9edb55360'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:24:51.633335 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-3ae4d8c0-27fb-499d-bf20-05a829e41ed3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc3cd4819cbf438fb2c5d4e9edb55360'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:24:51.695761 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-3ae4d8c0-27fb-499d-bf20-05a829e41ed3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc3cd4819cbf438fb2c5d4e9edb55360'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:24:51.703005 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-3ae4d8c0-27fb-499d-bf20-05a829e41ed3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc3cd4819cbf438fb2c5d4e9edb55360'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:24:51.709523 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1303/2583] 10.222.0.23 () {58 vars in 1223 bytes} [Sat Jun 18 05:24:51 2022] DELETE /identity/v3/users/cc3cd4819cbf438fb2c5d4e9edb55360 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:24:51.713966 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402d8cc1-5eac-4b94-a87b-a8a73d6da4d2 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:24:51.714421 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402d8cc1-5eac-4b94-a87b-a8a73d6da4d2 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:24:51.714548 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402d8cc1-5eac-4b94-a87b-a8a73d6da4d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:24:51.732926 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-402d8cc1-5eac-4b94-a87b-a8a73d6da4d2 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:24:51.734157 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-402d8cc1-5eac-4b94-a87b-a8a73d6da4d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:24:51.793667 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1281/2584] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:24:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:24:51.803369 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae4d8c0-27fb-499d-bf20-05a829e41ed3 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:24:51.804768 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-3ae4d8c0-27fb-499d-bf20-05a829e41ed3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:24:51.862031 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae4d8c0-27fb-499d-bf20-05a829e41ed3 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:24:51.865964 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d481100-6919-4fee-9e1f-99357a9428ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c4d7cde8f4ad9aa479bb09e325db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:24:51.866600 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d481100-6919-4fee-9e1f-99357a9428ea service neutron] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:24:51.866883 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d481100-6919-4fee-9e1f-99357a9428ea service neutron] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:24:51.867158 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d481100-6919-4fee-9e1f-99357a9428ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:24:51.876056 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d481100-6919-4fee-9e1f-99357a9428ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:24:51.877534 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d481100-6919-4fee-9e1f-99357a9428ea service neutron] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:24:51.887024 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1304/2585] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:24:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:24:52.275197 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402d8cc1-5eac-4b94-a87b-a8a73d6da4d2 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:24:52.277105 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-402d8cc1-5eac-4b94-a87b-a8a73d6da4d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:24:52.291785 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402d8cc1-5eac-4b94-a87b-a8a73d6da4d2 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:24:52.297690 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43aa3ab9-b973-4cc7-b2fb-d339ca60a8dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:24:52.298590 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43aa3ab9-b973-4cc7-b2fb-d339ca60a8dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:24:52.298876 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43aa3ab9-b973-4cc7-b2fb-d339ca60a8dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:24:52.299154 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43aa3ab9-b973-4cc7-b2fb-d339ca60a8dc admin admin] PATH_INFO: `/v3/projects/1e2a179f1f5e4593b295d282e34094a8` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:24:52.301863 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43aa3ab9-b973-4cc7-b2fb-d339ca60a8dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e2a179f1f5e4593b295d282e34094a8)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:24:52.304202 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43aa3ab9-b973-4cc7-b2fb-d339ca60a8dc admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:24:52.411897 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-43aa3ab9-b973-4cc7-b2fb-d339ca60a8dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e2a179f1f5e4593b295d282e34094a8', 'request_id': 'req-43aa3ab9-b973-4cc7-b2fb-d339ca60a8dc'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:24:52.417545 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-43aa3ab9-b973-4cc7-b2fb-d339ca60a8dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e2a179f1f5e4593b295d282e34094a8', 'request_id': 'req-43aa3ab9-b973-4cc7-b2fb-d339ca60a8dc'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:24:52.484150 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-43aa3ab9-b973-4cc7-b2fb-d339ca60a8dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e2a179f1f5e4593b295d282e34094a8', 'request_id': 'req-43aa3ab9-b973-4cc7-b2fb-d339ca60a8dc'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:24:52.496045 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-43aa3ab9-b973-4cc7-b2fb-d339ca60a8dc admin admin] The token cache is being invalidate because project 1e2a179f1f5e4593b295d282e34094a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80468) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 18 05:24:52.496331 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-43aa3ab9-b973-4cc7-b2fb-d339ca60a8dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:24:52.496969 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-43aa3ab9-b973-4cc7-b2fb-d339ca60a8dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:24:52.498535 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1282/2586] 10.222.0.23 () {58 vars in 1232 bytes} [Sat Jun 18 05:24:52 2022] DELETE /identity/v3/projects/1e2a179f1f5e4593b295d282e34094a8 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:24:52.502630 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d481100-6919-4fee-9e1f-99357a9428ea service neutron] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:24:52.513176 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d481100-6919-4fee-9e1f-99357a9428ea service neutron] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:24:52.531719 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:24:52.571215 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09141cc-848b-425d-8a50-7b794d96c3b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:24:52.572099 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09141cc-848b-425d-8a50-7b794d96c3b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:24:52.572379 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09141cc-848b-425d-8a50-7b794d96c3b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:24:52.572653 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09141cc-848b-425d-8a50-7b794d96c3b4 admin admin] PATH_INFO: `/v3/users/8c2a75fb3e2f49fd9b5fdc2b8d0bc1e6` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:24:52.573899 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09141cc-848b-425d-8a50-7b794d96c3b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c2a75fb3e2f49fd9b5fdc2b8d0bc1e6)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:24:52.575879 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09141cc-848b-425d-8a50-7b794d96c3b4 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:24:52.753908 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-b09141cc-848b-425d-8a50-7b794d96c3b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c2a75fb3e2f49fd9b5fdc2b8d0bc1e6'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:24:52.766120 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-b09141cc-848b-425d-8a50-7b794d96c3b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c2a75fb3e2f49fd9b5fdc2b8d0bc1e6'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:24:52.766824 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-b09141cc-848b-425d-8a50-7b794d96c3b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c2a75fb3e2f49fd9b5fdc2b8d0bc1e6'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:24:52.854814 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-b09141cc-848b-425d-8a50-7b794d96c3b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c2a75fb3e2f49fd9b5fdc2b8d0bc1e6'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:24:52.861973 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-b09141cc-848b-425d-8a50-7b794d96c3b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c2a75fb3e2f49fd9b5fdc2b8d0bc1e6'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:24:52.868158 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1305/2587] 10.222.0.23 () {58 vars in 1223 bytes} [Sat Jun 18 05:24:52 2022] DELETE /identity/v3/users/8c2a75fb3e2f49fd9b5fdc2b8d0bc1e6 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:24:53.309314 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43aa3ab9-b973-4cc7-b2fb-d339ca60a8dc admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:24:53.310842 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-43aa3ab9-b973-4cc7-b2fb-d339ca60a8dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:24:53.325460 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43aa3ab9-b973-4cc7-b2fb-d339ca60a8dc admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:24:53.384411 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d071d14b-8065-4cd6-81e9-2085361f3998 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:24:53.385227 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d071d14b-8065-4cd6-81e9-2085361f3998 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:24:53.385681 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d071d14b-8065-4cd6-81e9-2085361f3998 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:24:53.385962 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d071d14b-8065-4cd6-81e9-2085361f3998 admin admin] PATH_INFO: `/v3/projects/fa64432c05014fb5a8ab6a33254079d5` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:24:53.387163 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d071d14b-8065-4cd6-81e9-2085361f3998 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa64432c05014fb5a8ab6a33254079d5)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:24:53.389071 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d071d14b-8065-4cd6-81e9-2085361f3998 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:24:53.485577 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:24:53.502306 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-d071d14b-8065-4cd6-81e9-2085361f3998 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa64432c05014fb5a8ab6a33254079d5', 'request_id': 'req-d071d14b-8065-4cd6-81e9-2085361f3998'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:24:53.508333 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-d071d14b-8065-4cd6-81e9-2085361f3998 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa64432c05014fb5a8ab6a33254079d5', 'request_id': 'req-d071d14b-8065-4cd6-81e9-2085361f3998'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:24:53.602904 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-d071d14b-8065-4cd6-81e9-2085361f3998 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa64432c05014fb5a8ab6a33254079d5', 'request_id': 'req-d071d14b-8065-4cd6-81e9-2085361f3998'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:24:53.617085 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-d071d14b-8065-4cd6-81e9-2085361f3998 admin admin] The token cache is being invalidate because project fa64432c05014fb5a8ab6a33254079d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80468) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 18 05:24:53.617392 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-d071d14b-8065-4cd6-81e9-2085361f3998 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:24:53.618066 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-d071d14b-8065-4cd6-81e9-2085361f3998 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:24:53.619790 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1283/2588] 10.222.0.23 () {58 vars in 1232 bytes} [Sat Jun 18 05:24:53 2022] DELETE /identity/v3/projects/fa64432c05014fb5a8ab6a33254079d5 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:24:53.624837 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50513cc2-be70-46f9-9e44-d8f34bf30655 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:24:53.625407 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50513cc2-be70-46f9-9e44-d8f34bf30655 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:24:53.625407 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50513cc2-be70-46f9-9e44-d8f34bf30655 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:24:53.643572 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-50513cc2-be70-46f9-9e44-d8f34bf30655 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:24:53.644831 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-50513cc2-be70-46f9-9e44-d8f34bf30655 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:24:53.705826 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1306/2589] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:24:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:24:53.713871 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db25b3a-91df-4474-8e62-63313d41f72a None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:24:53.714163 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db25b3a-91df-4474-8e62-63313d41f72a None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:24:53.714441 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db25b3a-91df-4474-8e62-63313d41f72a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:24:53.733274 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-9db25b3a-91df-4474-8e62-63313d41f72a None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:24:53.734511 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-9db25b3a-91df-4474-8e62-63313d41f72a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:24:53.743620 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1284/2590] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:24:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:24:53.747749 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50513cc2-be70-46f9-9e44-d8f34bf30655 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:24:53.757644 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50513cc2-be70-46f9-9e44-d8f34bf30655 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:24:53.762300 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf695f40-8965-4298-bb02-fe993bc3240e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:24:53.762946 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf695f40-8965-4298-bb02-fe993bc3240e admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:24:53.763233 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf695f40-8965-4298-bb02-fe993bc3240e admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:24:53.763508 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf695f40-8965-4298-bb02-fe993bc3240e admin admin] PATH_INFO: `/v3/domains` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:24:53.764229 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf695f40-8965-4298-bb02-fe993bc3240e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 05:24:53.764540 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf695f40-8965-4298-bb02-fe993bc3240e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:24:53.766181 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf695f40-8965-4298-bb02-fe993bc3240e admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:24:53.777640 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1307/2591] 10.222.0.23 () {58 vars in 1152 bytes} [Sat Jun 18 05:24:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 05:24:53.782068 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db25b3a-91df-4474-8e62-63313d41f72a None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:24:53.792355 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db25b3a-91df-4474-8e62-63313d41f72a None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:24:53.797089 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774038fc-3b14-4240-9394-2db29e274d1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:24:53.800700 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774038fc-3b14-4240-9394-2db29e274d1f admin admin] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:24:53.800700 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774038fc-3b14-4240-9394-2db29e274d1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:24:53.800700 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774038fc-3b14-4240-9394-2db29e274d1f admin admin] PATH_INFO: `/v3/projects` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:24:53.800700 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774038fc-3b14-4240-9394-2db29e274d1f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:24:53.801705 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774038fc-3b14-4240-9394-2db29e274d1f admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:24:53.879426 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1285/2592] 10.222.0.23 () {60 vars in 1152 bytes} [Sat Jun 18 05:24:53 2022] POST /identity/v3/projects => generated 387 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 18 05:24:53.883290 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf695f40-8965-4298-bb02-fe993bc3240e admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:24:53.893346 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf695f40-8965-4298-bb02-fe993bc3240e admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:24:53.920074 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:24:53.950555 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425e352e-262a-408a-a59c-588e02cc79d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:24:53.951215 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425e352e-262a-408a-a59c-588e02cc79d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:24:53.951497 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425e352e-262a-408a-a59c-588e02cc79d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:24:53.951772 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425e352e-262a-408a-a59c-588e02cc79d2 admin admin] PATH_INFO: `/v3/users` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:24:53.952464 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425e352e-262a-408a-a59c-588e02cc79d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:24:53.954430 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425e352e-262a-408a-a59c-588e02cc79d2 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:24:54.049947 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1308/2593] 10.222.0.23 () {60 vars in 1143 bytes} [Sat Jun 18 05:24:53 2022] POST /identity/v3/users => generated 294 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:24:54.054133 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774038fc-3b14-4240-9394-2db29e274d1f admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:24:54.064598 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774038fc-3b14-4240-9394-2db29e274d1f admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:24:54.069451 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9146ce41-e1e5-48ba-a165-4a9f2dea2097 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:24:54.070098 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9146ce41-e1e5-48ba-a165-4a9f2dea2097 admin admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:24:54.070412 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9146ce41-e1e5-48ba-a165-4a9f2dea2097 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:24:54.070689 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9146ce41-e1e5-48ba-a165-4a9f2dea2097 admin admin] PATH_INFO: `/v3/roles` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:24:54.071444 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9146ce41-e1e5-48ba-a165-4a9f2dea2097 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:24:54.073059 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9146ce41-e1e5-48ba-a165-4a9f2dea2097 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:24:54.084329 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1286/2594] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:24:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:24:54.088212 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425e352e-262a-408a-a59c-588e02cc79d2 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:24:54.099667 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425e352e-262a-408a-a59c-588e02cc79d2 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:24:54.104367 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb7b059-f63f-4198-a301-c537b1a438ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:24:54.105011 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb7b059-f63f-4198-a301-c537b1a438ea admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:24:54.105295 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb7b059-f63f-4198-a301-c537b1a438ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:24:54.105607 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb7b059-f63f-4198-a301-c537b1a438ea admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:24:54.106329 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb7b059-f63f-4198-a301-c537b1a438ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:24:54.107964 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb7b059-f63f-4198-a301-c537b1a438ea admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:24:54.117766 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1309/2595] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:24:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:24:54.121654 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9146ce41-e1e5-48ba-a165-4a9f2dea2097 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:24:54.132020 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9146ce41-e1e5-48ba-a165-4a9f2dea2097 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:24:54.136602 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476e2a3e-96a5-4164-bc6a-be5ca9327569 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:24:54.137492 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476e2a3e-96a5-4164-bc6a-be5ca9327569 admin admin] REQUEST_METHOD: `PUT` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:24:54.137780 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476e2a3e-96a5-4164-bc6a-be5ca9327569 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:24:54.138057 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476e2a3e-96a5-4164-bc6a-be5ca9327569 admin admin] PATH_INFO: `/v3/projects/70af061887cc42a28960407dccb5657a/users/e9f61415d1ad46d593a467b9c59f30b8/roles/cbf5c4c26898413d93e73b46b41046a3` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:24:54.153150 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476e2a3e-96a5-4164-bc6a-be5ca9327569 admin admin] RBAC: Authorizing `identity:create_grant(project_id=70af061887cc42a28960407dccb5657a, user_id=e9f61415d1ad46d593a467b9c59f30b8, role_id=cbf5c4c26898413d93e73b46b41046a3)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:24:54.156069 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476e2a3e-96a5-4164-bc6a-be5ca9327569 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:24:54.227459 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1287/2596] 10.222.0.23 () {60 vars in 1482 bytes} [Sat Jun 18 05:24:54 2022] PUT /identity/v3/projects/70af061887cc42a28960407dccb5657a/users/e9f61415d1ad46d593a467b9c59f30b8/roles/cbf5c4c26898413d93e73b46b41046a3 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:24:54.237817 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27561a3f-6638-43fa-8e79-79ba8f5fb313 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:24:54.238102 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27561a3f-6638-43fa-8e79-79ba8f5fb313 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:24:54.238376 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27561a3f-6638-43fa-8e79-79ba8f5fb313 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:24:54.263842 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-27561a3f-6638-43fa-8e79-79ba8f5fb313 None None] MFA Rules not processed for user `e9f61415d1ad46d593a467b9c59f30b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:24:54.265033 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-27561a3f-6638-43fa-8e79-79ba8f5fb313 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:24:54.328186 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1310/2597] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:24:54 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:24:54.333281 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476e2a3e-96a5-4164-bc6a-be5ca9327569 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:24:54.394068 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476e2a3e-96a5-4164-bc6a-be5ca9327569 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:24:54.398724 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6448e18-66ec-4d58-bc4d-e5a89917d5bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:24:54.399366 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6448e18-66ec-4d58-bc4d-e5a89917d5bc admin admin] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:24:54.399643 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6448e18-66ec-4d58-bc4d-e5a89917d5bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:24:54.399915 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6448e18-66ec-4d58-bc4d-e5a89917d5bc admin admin] PATH_INFO: `/v3/projects` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:24:54.400608 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6448e18-66ec-4d58-bc4d-e5a89917d5bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:24:54.402533 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6448e18-66ec-4d58-bc4d-e5a89917d5bc admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:24:54.531904 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1288/2598] 10.222.0.23 () {60 vars in 1152 bytes} [Sat Jun 18 05:24:54 2022] POST /identity/v3/projects => generated 387 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:24:54.536106 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27561a3f-6638-43fa-8e79-79ba8f5fb313 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:24:54.546949 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27561a3f-6638-43fa-8e79-79ba8f5fb313 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:24:54.602800 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cb7428-c6e7-440b-9919-1ba7c359fd72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:24:54.603467 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cb7428-c6e7-440b-9919-1ba7c359fd72 admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:24:54.603757 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cb7428-c6e7-440b-9919-1ba7c359fd72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:24:54.604028 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cb7428-c6e7-440b-9919-1ba7c359fd72 admin admin] PATH_INFO: `/v3/users` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:24:54.604720 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cb7428-c6e7-440b-9919-1ba7c359fd72 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:24:54.606687 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cb7428-c6e7-440b-9919-1ba7c359fd72 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:24:54.684461 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1311/2599] 10.222.0.23 () {60 vars in 1143 bytes} [Sat Jun 18 05:24:54 2022] POST /identity/v3/users => generated 300 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:24:54.688506 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6448e18-66ec-4d58-bc4d-e5a89917d5bc admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:24:54.702002 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6448e18-66ec-4d58-bc4d-e5a89917d5bc admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:24:54.707925 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f74be49-a407-48d7-989e-bf5895086ae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:24:54.708578 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f74be49-a407-48d7-989e-bf5895086ae1 admin admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:24:54.708862 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f74be49-a407-48d7-989e-bf5895086ae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:24:54.709132 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f74be49-a407-48d7-989e-bf5895086ae1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:24:54.709896 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f74be49-a407-48d7-989e-bf5895086ae1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:24:54.711456 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f74be49-a407-48d7-989e-bf5895086ae1 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:24:54.723955 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1289/2600] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:24:54 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:24:54.728018 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cb7428-c6e7-440b-9919-1ba7c359fd72 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:24:54.738601 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cb7428-c6e7-440b-9919-1ba7c359fd72 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:24:54.743192 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b8896c-0073-46af-9d69-bed701d5e1b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:24:54.744057 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b8896c-0073-46af-9d69-bed701d5e1b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:24:54.744337 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b8896c-0073-46af-9d69-bed701d5e1b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:24:54.744609 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b8896c-0073-46af-9d69-bed701d5e1b0 admin admin] PATH_INFO: `/v3/projects/daec4c1e984040f29e1a6317901fc18f/users/59a05d9a7a624397a530f123156e189f/roles/21c65f2c51bd4dd283d13507d2714467` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:24:54.759943 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b8896c-0073-46af-9d69-bed701d5e1b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=daec4c1e984040f29e1a6317901fc18f, user_id=59a05d9a7a624397a530f123156e189f, role_id=21c65f2c51bd4dd283d13507d2714467)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:24:54.762658 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b8896c-0073-46af-9d69-bed701d5e1b0 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:24:54.821877 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1312/2601] 10.222.0.23 () {60 vars in 1482 bytes} [Sat Jun 18 05:24:54 2022] PUT /identity/v3/projects/daec4c1e984040f29e1a6317901fc18f/users/59a05d9a7a624397a530f123156e189f/roles/21c65f2c51bd4dd283d13507d2714467 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:24:54.825721 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f74be49-a407-48d7-989e-bf5895086ae1 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:24:54.835782 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f74be49-a407-48d7-989e-bf5895086ae1 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:24:54.855888 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:24:54.892905 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00284543-a62e-4b01-ab53-408d90f6b359 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:24:54.893618 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00284543-a62e-4b01-ab53-408d90f6b359 admin admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:24:54.893901 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00284543-a62e-4b01-ab53-408d90f6b359 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:24:54.894186 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00284543-a62e-4b01-ab53-408d90f6b359 admin admin] PATH_INFO: `/v3/roles` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:24:54.895089 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00284543-a62e-4b01-ab53-408d90f6b359 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:24:54.897362 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00284543-a62e-4b01-ab53-408d90f6b359 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:24:54.907334 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1290/2602] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:24:54 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:24:54.911408 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b8896c-0073-46af-9d69-bed701d5e1b0 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:24:54.916876 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:24:54.922143 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b8896c-0073-46af-9d69-bed701d5e1b0 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:24:54.927557 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28af2376-ac90-4ed3-b09a-07933b257d7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:24:54.928426 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28af2376-ac90-4ed3-b09a-07933b257d7b admin admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:24:54.928751 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28af2376-ac90-4ed3-b09a-07933b257d7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:24:54.929028 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28af2376-ac90-4ed3-b09a-07933b257d7b admin admin] PATH_INFO: `/v3/projects/daec4c1e984040f29e1a6317901fc18f/users/59a05d9a7a624397a530f123156e189f/roles/21c65f2c51bd4dd283d13507d2714467` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:24:54.931046 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28af2376-ac90-4ed3-b09a-07933b257d7b admin admin] RBAC: Authorizing `identity:create_grant(project_id=daec4c1e984040f29e1a6317901fc18f, user_id=59a05d9a7a624397a530f123156e189f, role_id=21c65f2c51bd4dd283d13507d2714467)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:24:54.933738 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28af2376-ac90-4ed3-b09a-07933b257d7b admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:24:54.944694 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1313/2603] 10.222.0.23 () {60 vars in 1482 bytes} [Sat Jun 18 05:24:54 2022] PUT /identity/v3/projects/daec4c1e984040f29e1a6317901fc18f/users/59a05d9a7a624397a530f123156e189f/roles/21c65f2c51bd4dd283d13507d2714467 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:24:54.955521 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb1b093-c55a-4536-90cf-ee12a8a841f9 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:24:54.956309 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb1b093-c55a-4536-90cf-ee12a8a841f9 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:24:54.956671 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb1b093-c55a-4536-90cf-ee12a8a841f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:24:54.984004 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-fbb1b093-c55a-4536-90cf-ee12a8a841f9 None None] MFA Rules not processed for user `59a05d9a7a624397a530f123156e189f`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:24:54.985211 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-fbb1b093-c55a-4536-90cf-ee12a8a841f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:24:55.052437 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1291/2604] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:24:54 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:24:55.063514 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28af2376-ac90-4ed3-b09a-07933b257d7b admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:24:55.065404 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-28af2376-ac90-4ed3-b09a-07933b257d7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:24:55.122278 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28af2376-ac90-4ed3-b09a-07933b257d7b admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:24:55.125837 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bb2bb3-6509-4cda-bd86-61f86a54a3d4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9de0428abc8400aa338aba4f7c34c94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:24:55.126465 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bb2bb3-6509-4cda-bd86-61f86a54a3d4 service cinder] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:24:55.126744 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bb2bb3-6509-4cda-bd86-61f86a54a3d4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:24:55.127017 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bb2bb3-6509-4cda-bd86-61f86a54a3d4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:24:55.136356 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bb2bb3-6509-4cda-bd86-61f86a54a3d4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:24:55.137809 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bb2bb3-6509-4cda-bd86-61f86a54a3d4 service cinder] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:24:55.147742 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1314/2605] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:24:55 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:24:55.174897 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb1b093-c55a-4536-90cf-ee12a8a841f9 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:24:55.237959 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb1b093-c55a-4536-90cf-ee12a8a841f9 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:24:55.242703 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67585b93-7a4f-4d9c-8263-6cdfdeeeeeaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:24:55.243558 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67585b93-7a4f-4d9c-8263-6cdfdeeeeeaa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:24:55.243842 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67585b93-7a4f-4d9c-8263-6cdfdeeeeeaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:24:55.244125 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67585b93-7a4f-4d9c-8263-6cdfdeeeeeaa admin admin] PATH_INFO: `/v3/users/e9f61415d1ad46d593a467b9c59f30b8` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:24:55.245353 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67585b93-7a4f-4d9c-8263-6cdfdeeeeeaa admin admin] RBAC: Authorizing `identity:delete_user(user_id=e9f61415d1ad46d593a467b9c59f30b8)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:24:55.247313 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67585b93-7a4f-4d9c-8263-6cdfdeeeeeaa admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:24:55.407195 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-67585b93-7a4f-4d9c-8263-6cdfdeeeeeaa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e9f61415d1ad46d593a467b9c59f30b8'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:24:55.407862 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-67585b93-7a4f-4d9c-8263-6cdfdeeeeeaa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e9f61415d1ad46d593a467b9c59f30b8'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:24:55.410073 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:24:55.506244 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-67585b93-7a4f-4d9c-8263-6cdfdeeeeeaa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e9f61415d1ad46d593a467b9c59f30b8'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:24:55.520297 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-67585b93-7a4f-4d9c-8263-6cdfdeeeeeaa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e9f61415d1ad46d593a467b9c59f30b8'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:24:55.527631 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-67585b93-7a4f-4d9c-8263-6cdfdeeeeeaa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e9f61415d1ad46d593a467b9c59f30b8'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:24:55.534218 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1292/2606] 10.222.0.23 () {58 vars in 1223 bytes} [Sat Jun 18 05:24:55 2022] DELETE /identity/v3/users/e9f61415d1ad46d593a467b9c59f30b8 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:24:55.538907 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e52d54-5c43-4081-9971-c6a020352ddd None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:24:55.539200 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e52d54-5c43-4081-9971-c6a020352ddd None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:24:55.539482 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e52d54-5c43-4081-9971-c6a020352ddd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:24:55.557965 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-f4e52d54-5c43-4081-9971-c6a020352ddd None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:24:55.559173 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-f4e52d54-5c43-4081-9971-c6a020352ddd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:24:55.619179 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1315/2607] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:24:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:24:55.629533 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67585b93-7a4f-4d9c-8263-6cdfdeeeeeaa admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:24:55.630961 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-67585b93-7a4f-4d9c-8263-6cdfdeeeeeaa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:24:55.688245 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67585b93-7a4f-4d9c-8263-6cdfdeeeeeaa admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:24:55.692198 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4b3c59-ae33-414e-a1f1-f9140158f049 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c4d7cde8f4ad9aa479bb09e325db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:24:55.692832 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4b3c59-ae33-414e-a1f1-f9140158f049 service neutron] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:24:55.693116 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4b3c59-ae33-414e-a1f1-f9140158f049 service neutron] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:24:55.693417 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4b3c59-ae33-414e-a1f1-f9140158f049 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:24:55.702270 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4b3c59-ae33-414e-a1f1-f9140158f049 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:24:55.703704 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4b3c59-ae33-414e-a1f1-f9140158f049 service neutron] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:24:55.713734 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1293/2608] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:24:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:24:56.125288 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e52d54-5c43-4081-9971-c6a020352ddd None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:24:56.127035 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-f4e52d54-5c43-4081-9971-c6a020352ddd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:24:56.142316 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e52d54-5c43-4081-9971-c6a020352ddd None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:24:56.147919 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697ec776-2207-44c1-bea2-0ef4bbb433a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:24:56.148816 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697ec776-2207-44c1-bea2-0ef4bbb433a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:24:56.149145 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697ec776-2207-44c1-bea2-0ef4bbb433a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:24:56.149525 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697ec776-2207-44c1-bea2-0ef4bbb433a7 admin admin] PATH_INFO: `/v3/projects/70af061887cc42a28960407dccb5657a` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:24:56.150721 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697ec776-2207-44c1-bea2-0ef4bbb433a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=70af061887cc42a28960407dccb5657a)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:24:56.157218 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697ec776-2207-44c1-bea2-0ef4bbb433a7 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:24:56.263491 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-697ec776-2207-44c1-bea2-0ef4bbb433a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '70af061887cc42a28960407dccb5657a', 'request_id': 'req-697ec776-2207-44c1-bea2-0ef4bbb433a7'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:24:56.268916 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-697ec776-2207-44c1-bea2-0ef4bbb433a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '70af061887cc42a28960407dccb5657a', 'request_id': 'req-697ec776-2207-44c1-bea2-0ef4bbb433a7'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:24:56.275423 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-697ec776-2207-44c1-bea2-0ef4bbb433a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '70af061887cc42a28960407dccb5657a', 'request_id': 'req-697ec776-2207-44c1-bea2-0ef4bbb433a7'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:24:56.346177 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-697ec776-2207-44c1-bea2-0ef4bbb433a7 admin admin] The token cache is being invalidate because project 70af061887cc42a28960407dccb5657a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80469) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 18 05:24:56.346469 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-697ec776-2207-44c1-bea2-0ef4bbb433a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:24:56.347102 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-697ec776-2207-44c1-bea2-0ef4bbb433a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:24:56.348742 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1316/2609] 10.222.0.23 () {58 vars in 1232 bytes} [Sat Jun 18 05:24:56 2022] DELETE /identity/v3/projects/70af061887cc42a28960407dccb5657a => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:24:56.352959 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4b3c59-ae33-414e-a1f1-f9140158f049 service neutron] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:24:56.362715 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4b3c59-ae33-414e-a1f1-f9140158f049 service neutron] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:24:56.420140 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbc888b-80ac-4cad-91b1-e84d955218ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:24:56.421000 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbc888b-80ac-4cad-91b1-e84d955218ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:24:56.421286 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbc888b-80ac-4cad-91b1-e84d955218ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:24:56.421609 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbc888b-80ac-4cad-91b1-e84d955218ae admin admin] PATH_INFO: `/v3/users/59a05d9a7a624397a530f123156e189f` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:24:56.422795 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbc888b-80ac-4cad-91b1-e84d955218ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=59a05d9a7a624397a530f123156e189f)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:24:56.424721 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbc888b-80ac-4cad-91b1-e84d955218ae admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:24:56.587666 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-dbbc888b-80ac-4cad-91b1-e84d955218ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '59a05d9a7a624397a530f123156e189f'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:24:56.588279 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-dbbc888b-80ac-4cad-91b1-e84d955218ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '59a05d9a7a624397a530f123156e189f'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:24:56.654499 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-dbbc888b-80ac-4cad-91b1-e84d955218ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '59a05d9a7a624397a530f123156e189f'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:24:56.666384 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:24:56.667682 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-dbbc888b-80ac-4cad-91b1-e84d955218ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '59a05d9a7a624397a530f123156e189f'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:24:56.674404 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-dbbc888b-80ac-4cad-91b1-e84d955218ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '59a05d9a7a624397a530f123156e189f'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:24:56.681587 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1294/2610] 10.222.0.23 () {58 vars in 1223 bytes} [Sat Jun 18 05:24:56 2022] DELETE /identity/v3/users/59a05d9a7a624397a530f123156e189f => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:24:57.074030 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697ec776-2207-44c1-bea2-0ef4bbb433a7 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:24:57.075507 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-697ec776-2207-44c1-bea2-0ef4bbb433a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:24:57.089479 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697ec776-2207-44c1-bea2-0ef4bbb433a7 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:24:57.103034 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:24:57.148599 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be430c7-06f5-4771-93d5-eb3d540de15a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:24:57.149504 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be430c7-06f5-4771-93d5-eb3d540de15a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:24:57.149789 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be430c7-06f5-4771-93d5-eb3d540de15a admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:24:57.150066 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be430c7-06f5-4771-93d5-eb3d540de15a admin admin] PATH_INFO: `/v3/projects/daec4c1e984040f29e1a6317901fc18f` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:24:57.151246 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be430c7-06f5-4771-93d5-eb3d540de15a admin admin] RBAC: Authorizing `identity:delete_project(project_id=daec4c1e984040f29e1a6317901fc18f)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:24:57.153208 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be430c7-06f5-4771-93d5-eb3d540de15a admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:24:57.281908 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-4be430c7-06f5-4771-93d5-eb3d540de15a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'daec4c1e984040f29e1a6317901fc18f', 'request_id': 'req-4be430c7-06f5-4771-93d5-eb3d540de15a'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:24:57.288040 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-4be430c7-06f5-4771-93d5-eb3d540de15a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'daec4c1e984040f29e1a6317901fc18f', 'request_id': 'req-4be430c7-06f5-4771-93d5-eb3d540de15a'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:24:57.294408 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-4be430c7-06f5-4771-93d5-eb3d540de15a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'daec4c1e984040f29e1a6317901fc18f', 'request_id': 'req-4be430c7-06f5-4771-93d5-eb3d540de15a'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:24:57.381899 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-4be430c7-06f5-4771-93d5-eb3d540de15a admin admin] The token cache is being invalidate because project daec4c1e984040f29e1a6317901fc18f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80469) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 18 05:24:57.382217 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-4be430c7-06f5-4771-93d5-eb3d540de15a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:24:57.382902 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-4be430c7-06f5-4771-93d5-eb3d540de15a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:24:57.384426 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1317/2611] 10.222.0.23 () {58 vars in 1232 bytes} [Sat Jun 18 05:24:57 2022] DELETE /identity/v3/projects/daec4c1e984040f29e1a6317901fc18f => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:24:57.390181 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3102fe7-3c66-4533-99db-0d39ece48393 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:24:57.390473 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3102fe7-3c66-4533-99db-0d39ece48393 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:24:57.390755 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3102fe7-3c66-4533-99db-0d39ece48393 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:24:57.410336 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-f3102fe7-3c66-4533-99db-0d39ece48393 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:24:57.411642 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-f3102fe7-3c66-4533-99db-0d39ece48393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:24:57.476928 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1295/2612] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:24:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:24:57.484642 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb1e51d-3d13-4d61-bb75-42b03c2633ce None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:24:57.484934 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb1e51d-3d13-4d61-bb75-42b03c2633ce None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:24:57.485234 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb1e51d-3d13-4d61-bb75-42b03c2633ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:24:57.504900 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-adb1e51d-3d13-4d61-bb75-42b03c2633ce None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:24:57.506199 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-adb1e51d-3d13-4d61-bb75-42b03c2633ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:24:57.515665 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1318/2613] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:24:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:24:57.520676 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3102fe7-3c66-4533-99db-0d39ece48393 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:24:57.531012 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3102fe7-3c66-4533-99db-0d39ece48393 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:24:57.537276 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c5a788-9194-45e7-8d60-56e80e88dd6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:24:57.537969 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c5a788-9194-45e7-8d60-56e80e88dd6e admin admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:24:57.538349 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c5a788-9194-45e7-8d60-56e80e88dd6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:24:57.538624 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c5a788-9194-45e7-8d60-56e80e88dd6e admin admin] PATH_INFO: `/v3/domains` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:24:57.539352 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c5a788-9194-45e7-8d60-56e80e88dd6e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 05:24:57.539664 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c5a788-9194-45e7-8d60-56e80e88dd6e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:24:57.541322 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c5a788-9194-45e7-8d60-56e80e88dd6e admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:24:57.553823 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1296/2614] 10.222.0.23 () {58 vars in 1152 bytes} [Sat Jun 18 05:24:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 05:24:57.558171 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb1e51d-3d13-4d61-bb75-42b03c2633ce None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:24:57.568091 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb1e51d-3d13-4d61-bb75-42b03c2633ce None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:24:57.572807 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc258485-a2ef-4b5b-8e56-ea510d2d7e5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:24:57.573480 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc258485-a2ef-4b5b-8e56-ea510d2d7e5d admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:24:57.573768 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc258485-a2ef-4b5b-8e56-ea510d2d7e5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:24:57.574079 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc258485-a2ef-4b5b-8e56-ea510d2d7e5d admin admin] PATH_INFO: `/v3/projects` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:24:57.574771 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc258485-a2ef-4b5b-8e56-ea510d2d7e5d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:24:57.576683 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc258485-a2ef-4b5b-8e56-ea510d2d7e5d admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:24:57.652277 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1319/2615] 10.222.0.23 () {60 vars in 1152 bytes} [Sat Jun 18 05:24:57 2022] POST /identity/v3/projects => generated 383 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:24:57.656527 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c5a788-9194-45e7-8d60-56e80e88dd6e admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:24:57.667044 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c5a788-9194-45e7-8d60-56e80e88dd6e admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:24:57.726179 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134410ed-708f-4d03-8af6-03a6a9223895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:24:57.726874 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134410ed-708f-4d03-8af6-03a6a9223895 admin admin] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:24:57.727159 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134410ed-708f-4d03-8af6-03a6a9223895 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:24:57.727432 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134410ed-708f-4d03-8af6-03a6a9223895 admin admin] PATH_INFO: `/v3/users` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:24:57.728130 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134410ed-708f-4d03-8af6-03a6a9223895 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:24:57.730123 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134410ed-708f-4d03-8af6-03a6a9223895 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:24:57.814892 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1297/2616] 10.222.0.23 () {60 vars in 1143 bytes} [Sat Jun 18 05:24:57 2022] POST /identity/v3/users => generated 292 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:24:57.819693 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc258485-a2ef-4b5b-8e56-ea510d2d7e5d admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:24:57.830501 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc258485-a2ef-4b5b-8e56-ea510d2d7e5d admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:24:57.836071 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9719d5-6c37-4469-bdf3-aa744c9bcf74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:24:57.836729 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9719d5-6c37-4469-bdf3-aa744c9bcf74 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:24:57.840726 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9719d5-6c37-4469-bdf3-aa744c9bcf74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:24:57.840726 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9719d5-6c37-4469-bdf3-aa744c9bcf74 admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:24:57.840726 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9719d5-6c37-4469-bdf3-aa744c9bcf74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:24:57.840726 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9719d5-6c37-4469-bdf3-aa744c9bcf74 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:24:57.852005 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1320/2617] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:24:57 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 18 05:24:57.856121 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134410ed-708f-4d03-8af6-03a6a9223895 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:24:57.869601 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134410ed-708f-4d03-8af6-03a6a9223895 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:24:57.871571 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d25f985-e55a-4280-bdfb-0210bf91e163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:24:57.872273 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d25f985-e55a-4280-bdfb-0210bf91e163 admin admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:24:57.872593 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d25f985-e55a-4280-bdfb-0210bf91e163 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:24:57.872873 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d25f985-e55a-4280-bdfb-0210bf91e163 admin admin] PATH_INFO: `/v3/roles` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:24:57.873624 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d25f985-e55a-4280-bdfb-0210bf91e163 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:24:57.875275 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d25f985-e55a-4280-bdfb-0210bf91e163 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:24:57.888733 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1298/2618] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:24:57 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 18 05:24:57.892558 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9719d5-6c37-4469-bdf3-aa744c9bcf74 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:24:57.902841 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9719d5-6c37-4469-bdf3-aa744c9bcf74 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:24:57.907729 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027fd7f2-9c55-473e-bcac-3fe8de58cbde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:24:57.908627 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027fd7f2-9c55-473e-bcac-3fe8de58cbde admin admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:24:57.908908 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027fd7f2-9c55-473e-bcac-3fe8de58cbde admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:24:57.909181 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027fd7f2-9c55-473e-bcac-3fe8de58cbde admin admin] PATH_INFO: `/v3/projects/62d28e3004094c23ade06b001bd2be7e/users/765ef416f53b4e7caec661b10ec08f09/roles/cbf5c4c26898413d93e73b46b41046a3` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:24:57.926662 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027fd7f2-9c55-473e-bcac-3fe8de58cbde admin admin] RBAC: Authorizing `identity:create_grant(project_id=62d28e3004094c23ade06b001bd2be7e, user_id=765ef416f53b4e7caec661b10ec08f09, role_id=cbf5c4c26898413d93e73b46b41046a3)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:24:57.928311 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027fd7f2-9c55-473e-bcac-3fe8de58cbde admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:24:57.998263 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1321/2619] 10.222.0.23 () {60 vars in 1482 bytes} [Sat Jun 18 05:24:57 2022] PUT /identity/v3/projects/62d28e3004094c23ade06b001bd2be7e/users/765ef416f53b4e7caec661b10ec08f09/roles/cbf5c4c26898413d93e73b46b41046a3 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 18 05:24:58.009150 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdd1f28-75c3-4200-8d7a-95e14f5150fd None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:24:58.009472 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdd1f28-75c3-4200-8d7a-95e14f5150fd None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:24:58.009760 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdd1f28-75c3-4200-8d7a-95e14f5150fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:24:58.036466 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-4cdd1f28-75c3-4200-8d7a-95e14f5150fd None None] MFA Rules not processed for user `765ef416f53b4e7caec661b10ec08f09`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:24:58.043227 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-4cdd1f28-75c3-4200-8d7a-95e14f5150fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:24:58.079965 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:24:58.102967 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1299/2620] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:24:58 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:24:58.107437 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027fd7f2-9c55-473e-bcac-3fe8de58cbde admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:24:58.154453 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:24:58.171998 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027fd7f2-9c55-473e-bcac-3fe8de58cbde admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:24:58.177319 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7e3141-c7c3-4e80-b446-f94515ae3227 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:24:58.177997 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7e3141-c7c3-4e80-b446-f94515ae3227 admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:24:58.178280 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7e3141-c7c3-4e80-b446-f94515ae3227 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:24:58.178555 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7e3141-c7c3-4e80-b446-f94515ae3227 admin admin] PATH_INFO: `/v3/projects` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:24:58.179246 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7e3141-c7c3-4e80-b446-f94515ae3227 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:24:58.181163 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7e3141-c7c3-4e80-b446-f94515ae3227 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:24:58.256947 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1322/2621] 10.222.0.23 () {60 vars in 1152 bytes} [Sat Jun 18 05:24:58 2022] POST /identity/v3/projects => generated 381 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:24:58.260786 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdd1f28-75c3-4200-8d7a-95e14f5150fd None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:24:58.271905 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdd1f28-75c3-4200-8d7a-95e14f5150fd None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:24:58.330282 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db909e6-4d62-4a39-8319-d5e2ebac8d81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:24:58.330919 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db909e6-4d62-4a39-8319-d5e2ebac8d81 admin admin] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:24:58.331215 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db909e6-4d62-4a39-8319-d5e2ebac8d81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:24:58.331478 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db909e6-4d62-4a39-8319-d5e2ebac8d81 admin admin] PATH_INFO: `/v3/users` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:24:58.332292 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db909e6-4d62-4a39-8319-d5e2ebac8d81 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:24:58.334341 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db909e6-4d62-4a39-8319-d5e2ebac8d81 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:24:58.400511 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1300/2622] 10.222.0.23 () {60 vars in 1143 bytes} [Sat Jun 18 05:24:58 2022] POST /identity/v3/users => generated 297 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:24:58.404622 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7e3141-c7c3-4e80-b446-f94515ae3227 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:24:58.416419 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7e3141-c7c3-4e80-b446-f94515ae3227 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:24:58.420984 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c078f0e0-0fec-44f6-bd22-9f49101d1199 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:24:58.421676 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c078f0e0-0fec-44f6-bd22-9f49101d1199 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:24:58.421956 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c078f0e0-0fec-44f6-bd22-9f49101d1199 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:24:58.422237 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c078f0e0-0fec-44f6-bd22-9f49101d1199 admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:24:58.423006 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c078f0e0-0fec-44f6-bd22-9f49101d1199 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:24:58.424594 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c078f0e0-0fec-44f6-bd22-9f49101d1199 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:24:58.435997 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1323/2623] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:24:58 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:24:58.440019 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db909e6-4d62-4a39-8319-d5e2ebac8d81 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:24:58.451108 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db909e6-4d62-4a39-8319-d5e2ebac8d81 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:24:58.456042 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535b68b7-d1c5-41a4-b142-5e738e6d83f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:24:58.456965 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535b68b7-d1c5-41a4-b142-5e738e6d83f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:24:58.457248 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535b68b7-d1c5-41a4-b142-5e738e6d83f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:24:58.457572 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535b68b7-d1c5-41a4-b142-5e738e6d83f5 admin admin] PATH_INFO: `/v3/projects/2da84fa2a7aa40c6802531f0025245e7/users/aba203bde1e54b78a0d4bd0a01796b1e/roles/21c65f2c51bd4dd283d13507d2714467` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:24:58.472928 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535b68b7-d1c5-41a4-b142-5e738e6d83f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2da84fa2a7aa40c6802531f0025245e7, user_id=aba203bde1e54b78a0d4bd0a01796b1e, role_id=21c65f2c51bd4dd283d13507d2714467)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:24:58.476924 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535b68b7-d1c5-41a4-b142-5e738e6d83f5 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:24:58.524705 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1301/2624] 10.222.0.23 () {60 vars in 1482 bytes} [Sat Jun 18 05:24:58 2022] PUT /identity/v3/projects/2da84fa2a7aa40c6802531f0025245e7/users/aba203bde1e54b78a0d4bd0a01796b1e/roles/21c65f2c51bd4dd283d13507d2714467 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:24:58.528738 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c078f0e0-0fec-44f6-bd22-9f49101d1199 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:24:58.539970 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c078f0e0-0fec-44f6-bd22-9f49101d1199 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:24:58.595327 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62a4fa6-e176-4cd9-8359-69cefd1e842c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:24:58.595978 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62a4fa6-e176-4cd9-8359-69cefd1e842c admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:24:58.596282 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62a4fa6-e176-4cd9-8359-69cefd1e842c admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:24:58.596552 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62a4fa6-e176-4cd9-8359-69cefd1e842c admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:24:58.597260 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62a4fa6-e176-4cd9-8359-69cefd1e842c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:24:58.599747 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62a4fa6-e176-4cd9-8359-69cefd1e842c admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:24:58.609607 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1324/2625] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:24:58 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:24:58.613704 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535b68b7-d1c5-41a4-b142-5e738e6d83f5 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:24:58.624137 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535b68b7-d1c5-41a4-b142-5e738e6d83f5 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:24:58.628778 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0250825-e227-4e41-887d-97db1bd182f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:24:58.629681 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0250825-e227-4e41-887d-97db1bd182f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:24:58.629984 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0250825-e227-4e41-887d-97db1bd182f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:24:58.630327 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0250825-e227-4e41-887d-97db1bd182f8 admin admin] PATH_INFO: `/v3/projects/2da84fa2a7aa40c6802531f0025245e7/users/aba203bde1e54b78a0d4bd0a01796b1e/roles/21c65f2c51bd4dd283d13507d2714467` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:24:58.632448 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0250825-e227-4e41-887d-97db1bd182f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2da84fa2a7aa40c6802531f0025245e7, user_id=aba203bde1e54b78a0d4bd0a01796b1e, role_id=21c65f2c51bd4dd283d13507d2714467)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:24:58.635204 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0250825-e227-4e41-887d-97db1bd182f8 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:24:58.645229 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1302/2626] 10.222.0.23 () {60 vars in 1482 bytes} [Sat Jun 18 05:24:58 2022] PUT /identity/v3/projects/2da84fa2a7aa40c6802531f0025245e7/users/aba203bde1e54b78a0d4bd0a01796b1e/roles/21c65f2c51bd4dd283d13507d2714467 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:24:58.657165 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9ccd61-9808-409c-ad8e-4c83611226e1 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:24:58.657416 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9ccd61-9808-409c-ad8e-4c83611226e1 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:24:58.657697 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9ccd61-9808-409c-ad8e-4c83611226e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:24:58.684982 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-dd9ccd61-9808-409c-ad8e-4c83611226e1 None None] MFA Rules not processed for user `aba203bde1e54b78a0d4bd0a01796b1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:24:58.686232 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-dd9ccd61-9808-409c-ad8e-4c83611226e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:24:58.752457 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1325/2627] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:24:58 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:24:58.763227 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0250825-e227-4e41-887d-97db1bd182f8 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:24:58.764886 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-b0250825-e227-4e41-887d-97db1bd182f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:24:58.822268 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0250825-e227-4e41-887d-97db1bd182f8 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:24:58.825952 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12615c7-bc4f-49d8-9c2b-ae9927c23586 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9de0428abc8400aa338aba4f7c34c94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:24:58.826582 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12615c7-bc4f-49d8-9c2b-ae9927c23586 service cinder] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:24:58.826865 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12615c7-bc4f-49d8-9c2b-ae9927c23586 service cinder] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:24:58.827141 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12615c7-bc4f-49d8-9c2b-ae9927c23586 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:24:58.837258 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12615c7-bc4f-49d8-9c2b-ae9927c23586 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:24:58.838759 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12615c7-bc4f-49d8-9c2b-ae9927c23586 service cinder] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:24:58.848175 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1303/2628] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:24:58 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:24:58.947375 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9ccd61-9808-409c-ad8e-4c83611226e1 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:24:58.957145 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9ccd61-9808-409c-ad8e-4c83611226e1 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:24:58.963645 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760f48cd-c4ae-4320-b658-792a959a1b0e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9de0428abc8400aa338aba4f7c34c94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:24:58.964280 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760f48cd-c4ae-4320-b658-792a959a1b0e service cinder] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:24:58.964579 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760f48cd-c4ae-4320-b658-792a959a1b0e service cinder] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:24:58.964845 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760f48cd-c4ae-4320-b658-792a959a1b0e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:24:58.998240 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:24:59.022097 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760f48cd-c4ae-4320-b658-792a959a1b0e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:24:59.023586 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760f48cd-c4ae-4320-b658-792a959a1b0e service cinder] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:24:59.032827 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1326/2629] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:24:58 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:25:03.195146 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12615c7-bc4f-49d8-9c2b-ae9927c23586 service cinder] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:25:03.237211 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:25:03.259539 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12615c7-bc4f-49d8-9c2b-ae9927c23586 service cinder] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:25:03.264728 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b66e266-7539-4703-be84-836dee98abbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:25:03.265744 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b66e266-7539-4703-be84-836dee98abbc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:25:03.266092 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b66e266-7539-4703-be84-836dee98abbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:25:03.266428 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b66e266-7539-4703-be84-836dee98abbc admin admin] PATH_INFO: `/v3/users/765ef416f53b4e7caec661b10ec08f09` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:25:03.267701 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b66e266-7539-4703-be84-836dee98abbc admin admin] RBAC: Authorizing `identity:delete_user(user_id=765ef416f53b4e7caec661b10ec08f09)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:25:03.269866 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b66e266-7539-4703-be84-836dee98abbc admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:25:03.480403 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-1b66e266-7539-4703-be84-836dee98abbc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '765ef416f53b4e7caec661b10ec08f09'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:25:03.480970 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-1b66e266-7539-4703-be84-836dee98abbc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '765ef416f53b4e7caec661b10ec08f09'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:25:03.564012 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-1b66e266-7539-4703-be84-836dee98abbc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '765ef416f53b4e7caec661b10ec08f09'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:25:03.580591 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-1b66e266-7539-4703-be84-836dee98abbc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '765ef416f53b4e7caec661b10ec08f09'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:25:03.589096 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-1b66e266-7539-4703-be84-836dee98abbc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '765ef416f53b4e7caec661b10ec08f09'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:25:03.595475 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1304/2630] 10.222.0.23 () {58 vars in 1223 bytes} [Sat Jun 18 05:25:03 2022] DELETE /identity/v3/users/765ef416f53b4e7caec661b10ec08f09 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:25:03.600331 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6f1d69-930b-49f9-9485-5741a3482766 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:25:03.600855 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6f1d69-930b-49f9-9485-5741a3482766 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:25:03.600970 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6f1d69-930b-49f9-9485-5741a3482766 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:25:03.617816 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-fa6f1d69-930b-49f9-9485-5741a3482766 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:25:03.619022 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-fa6f1d69-930b-49f9-9485-5741a3482766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:25:03.680711 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1327/2631] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:25:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:25:03.690579 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b66e266-7539-4703-be84-836dee98abbc admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:25:03.691972 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-1b66e266-7539-4703-be84-836dee98abbc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:25:03.748530 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b66e266-7539-4703-be84-836dee98abbc admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:25:03.752120 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc3be69-49e8-494a-b129-030584e7314f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c4d7cde8f4ad9aa479bb09e325db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:25:03.752915 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc3be69-49e8-494a-b129-030584e7314f service neutron] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:25:03.753204 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc3be69-49e8-494a-b129-030584e7314f service neutron] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:25:03.753529 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc3be69-49e8-494a-b129-030584e7314f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:25:03.763510 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc3be69-49e8-494a-b129-030584e7314f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:25:03.769563 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc3be69-49e8-494a-b129-030584e7314f service neutron] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:25:03.774883 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1305/2632] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:25:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:25:04.183818 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6f1d69-930b-49f9-9485-5741a3482766 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:25:04.185251 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-fa6f1d69-930b-49f9-9485-5741a3482766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:25:04.199128 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6f1d69-930b-49f9-9485-5741a3482766 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:25:04.204133 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0402cf1b-e411-463f-992c-d53f989c1d56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:25:04.211028 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0402cf1b-e411-463f-992c-d53f989c1d56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:25:04.211028 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0402cf1b-e411-463f-992c-d53f989c1d56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:25:04.211028 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0402cf1b-e411-463f-992c-d53f989c1d56 admin admin] PATH_INFO: `/v3/projects/62d28e3004094c23ade06b001bd2be7e` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:25:04.211028 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0402cf1b-e411-463f-992c-d53f989c1d56 admin admin] RBAC: Authorizing `identity:delete_project(project_id=62d28e3004094c23ade06b001bd2be7e)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:25:04.211028 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0402cf1b-e411-463f-992c-d53f989c1d56 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:25:04.337288 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-0402cf1b-e411-463f-992c-d53f989c1d56 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '62d28e3004094c23ade06b001bd2be7e', 'request_id': 'req-0402cf1b-e411-463f-992c-d53f989c1d56'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:25:04.342892 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-0402cf1b-e411-463f-992c-d53f989c1d56 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '62d28e3004094c23ade06b001bd2be7e', 'request_id': 'req-0402cf1b-e411-463f-992c-d53f989c1d56'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:25:04.350445 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-0402cf1b-e411-463f-992c-d53f989c1d56 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '62d28e3004094c23ade06b001bd2be7e', 'request_id': 'req-0402cf1b-e411-463f-992c-d53f989c1d56'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:25:04.432494 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-0402cf1b-e411-463f-992c-d53f989c1d56 admin admin] The token cache is being invalidate because project 62d28e3004094c23ade06b001bd2be7e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80469) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 18 05:25:04.432780 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-0402cf1b-e411-463f-992c-d53f989c1d56 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:25:04.433443 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-0402cf1b-e411-463f-992c-d53f989c1d56 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:25:04.434973 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1328/2633] 10.222.0.23 () {58 vars in 1232 bytes} [Sat Jun 18 05:25:04 2022] DELETE /identity/v3/projects/62d28e3004094c23ade06b001bd2be7e => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:25:04.439099 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc3be69-49e8-494a-b129-030584e7314f service neutron] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:25:04.448770 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc3be69-49e8-494a-b129-030584e7314f service neutron] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:25:04.504778 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1d04c6-66ea-44cc-93b4-8c8028fbcf90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:25:04.505684 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1d04c6-66ea-44cc-93b4-8c8028fbcf90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:25:04.506154 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1d04c6-66ea-44cc-93b4-8c8028fbcf90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:25:04.506436 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1d04c6-66ea-44cc-93b4-8c8028fbcf90 admin admin] PATH_INFO: `/v3/users/aba203bde1e54b78a0d4bd0a01796b1e` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:25:04.507650 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1d04c6-66ea-44cc-93b4-8c8028fbcf90 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aba203bde1e54b78a0d4bd0a01796b1e)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:25:04.509603 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1d04c6-66ea-44cc-93b4-8c8028fbcf90 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:25:04.511161 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:25:04.697074 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-ca1d04c6-66ea-44cc-93b4-8c8028fbcf90 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aba203bde1e54b78a0d4bd0a01796b1e'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:25:04.697746 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-ca1d04c6-66ea-44cc-93b4-8c8028fbcf90 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aba203bde1e54b78a0d4bd0a01796b1e'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:25:04.769188 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-ca1d04c6-66ea-44cc-93b4-8c8028fbcf90 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aba203bde1e54b78a0d4bd0a01796b1e'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:25:04.783156 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-ca1d04c6-66ea-44cc-93b4-8c8028fbcf90 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aba203bde1e54b78a0d4bd0a01796b1e'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:25:04.791210 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-ca1d04c6-66ea-44cc-93b4-8c8028fbcf90 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aba203bde1e54b78a0d4bd0a01796b1e'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:25:04.797338 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1306/2634] 10.222.0.23 () {58 vars in 1223 bytes} [Sat Jun 18 05:25:04 2022] DELETE /identity/v3/users/aba203bde1e54b78a0d4bd0a01796b1e => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:25:05.216919 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0402cf1b-e411-463f-992c-d53f989c1d56 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:25:05.218434 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-0402cf1b-e411-463f-992c-d53f989c1d56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:25:05.231798 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0402cf1b-e411-463f-992c-d53f989c1d56 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:25:05.290605 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1240c119-9348-4839-9e35-9889230f1894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:25:05.291460 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1240c119-9348-4839-9e35-9889230f1894 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:25:05.291746 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1240c119-9348-4839-9e35-9889230f1894 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:25:05.292022 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1240c119-9348-4839-9e35-9889230f1894 admin admin] PATH_INFO: `/v3/projects/2da84fa2a7aa40c6802531f0025245e7` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:25:05.293210 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1240c119-9348-4839-9e35-9889230f1894 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2da84fa2a7aa40c6802531f0025245e7)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:25:05.295188 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1240c119-9348-4839-9e35-9889230f1894 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:25:05.296622 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:25:05.382951 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-1240c119-9348-4839-9e35-9889230f1894 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2da84fa2a7aa40c6802531f0025245e7', 'request_id': 'req-1240c119-9348-4839-9e35-9889230f1894'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:25:05.388885 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-1240c119-9348-4839-9e35-9889230f1894 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2da84fa2a7aa40c6802531f0025245e7', 'request_id': 'req-1240c119-9348-4839-9e35-9889230f1894'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:25:05.395270 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-1240c119-9348-4839-9e35-9889230f1894 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2da84fa2a7aa40c6802531f0025245e7', 'request_id': 'req-1240c119-9348-4839-9e35-9889230f1894'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:25:05.475757 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-1240c119-9348-4839-9e35-9889230f1894 admin admin] The token cache is being invalidate because project 2da84fa2a7aa40c6802531f0025245e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80469) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 18 05:25:05.475961 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-1240c119-9348-4839-9e35-9889230f1894 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:25:05.476597 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-1240c119-9348-4839-9e35-9889230f1894 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:25:05.478239 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1329/2635] 10.222.0.23 () {58 vars in 1232 bytes} [Sat Jun 18 05:25:05 2022] DELETE /identity/v3/projects/2da84fa2a7aa40c6802531f0025245e7 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:25:05.483110 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c15d48-2dad-4973-9e32-3ee8af6b4dc8 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:25:05.483684 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c15d48-2dad-4973-9e32-3ee8af6b4dc8 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:25:05.483684 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c15d48-2dad-4973-9e32-3ee8af6b4dc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:25:05.502151 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-00c15d48-2dad-4973-9e32-3ee8af6b4dc8 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:25:05.503361 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-00c15d48-2dad-4973-9e32-3ee8af6b4dc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:25:05.563163 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1307/2636] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:25:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:25:05.570996 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c6df1d-fabe-4f07-800e-7fb7496e21c1 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:25:05.571285 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c6df1d-fabe-4f07-800e-7fb7496e21c1 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:25:05.571560 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c6df1d-fabe-4f07-800e-7fb7496e21c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:25:05.589868 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-58c6df1d-fabe-4f07-800e-7fb7496e21c1 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:25:05.591073 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-58c6df1d-fabe-4f07-800e-7fb7496e21c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:25:05.599539 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1330/2637] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:25:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:25:05.603585 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c15d48-2dad-4973-9e32-3ee8af6b4dc8 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:25:05.613354 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c15d48-2dad-4973-9e32-3ee8af6b4dc8 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:25:05.619242 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec6f8ec-29f3-4bca-be13-e71e5c83c810 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:25:05.619918 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec6f8ec-29f3-4bca-be13-e71e5c83c810 admin admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:25:05.620198 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec6f8ec-29f3-4bca-be13-e71e5c83c810 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:25:05.620473 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec6f8ec-29f3-4bca-be13-e71e5c83c810 admin admin] PATH_INFO: `/v3/domains` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:25:05.621189 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec6f8ec-29f3-4bca-be13-e71e5c83c810 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 05:25:05.621543 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec6f8ec-29f3-4bca-be13-e71e5c83c810 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:25:05.623197 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec6f8ec-29f3-4bca-be13-e71e5c83c810 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:25:05.635444 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1308/2638] 10.222.0.23 () {58 vars in 1152 bytes} [Sat Jun 18 05:25:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 18 05:25:05.639551 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c6df1d-fabe-4f07-800e-7fb7496e21c1 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:25:05.653107 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c6df1d-fabe-4f07-800e-7fb7496e21c1 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:25:05.658081 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cefb97b-c392-4ebb-8137-373d07c5b37f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:25:05.658727 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cefb97b-c392-4ebb-8137-373d07c5b37f admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:25:05.659009 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cefb97b-c392-4ebb-8137-373d07c5b37f admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:25:05.659286 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cefb97b-c392-4ebb-8137-373d07c5b37f admin admin] PATH_INFO: `/v3/projects` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:25:05.659988 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cefb97b-c392-4ebb-8137-373d07c5b37f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:25:05.662444 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cefb97b-c392-4ebb-8137-373d07c5b37f admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:25:05.749712 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1331/2639] 10.222.0.23 () {60 vars in 1152 bytes} [Sat Jun 18 05:25:05 2022] POST /identity/v3/projects => generated 407 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:25:05.749970 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec6f8ec-29f3-4bca-be13-e71e5c83c810 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:25:05.757924 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec6f8ec-29f3-4bca-be13-e71e5c83c810 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:25:05.813989 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97961ed-ac3c-43d2-a98e-c111b7388d15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:25:05.814644 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97961ed-ac3c-43d2-a98e-c111b7388d15 admin admin] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:25:05.814924 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97961ed-ac3c-43d2-a98e-c111b7388d15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:25:05.815196 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97961ed-ac3c-43d2-a98e-c111b7388d15 admin admin] PATH_INFO: `/v3/users` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:25:05.815869 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97961ed-ac3c-43d2-a98e-c111b7388d15 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:25:05.817839 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97961ed-ac3c-43d2-a98e-c111b7388d15 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:25:05.827215 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:25:05.929517 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1309/2640] 10.222.0.23 () {60 vars in 1143 bytes} [Sat Jun 18 05:25:05 2022] POST /identity/v3/users => generated 304 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:25:05.933930 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cefb97b-c392-4ebb-8137-373d07c5b37f admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:25:05.944332 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cefb97b-c392-4ebb-8137-373d07c5b37f admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:25:05.949001 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0177bc-6b13-447e-9dcc-38cde7f2cd65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:25:05.949701 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0177bc-6b13-447e-9dcc-38cde7f2cd65 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:25:05.950037 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0177bc-6b13-447e-9dcc-38cde7f2cd65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:25:05.950286 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0177bc-6b13-447e-9dcc-38cde7f2cd65 admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:25:05.951009 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0177bc-6b13-447e-9dcc-38cde7f2cd65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:25:05.952620 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0177bc-6b13-447e-9dcc-38cde7f2cd65 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:25:05.963531 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1332/2641] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:25:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:25:05.970114 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97961ed-ac3c-43d2-a98e-c111b7388d15 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:25:05.980745 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97961ed-ac3c-43d2-a98e-c111b7388d15 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:25:05.985723 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9ed5be-0241-40f2-8487-a8dfde69fdfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:25:05.986851 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9ed5be-0241-40f2-8487-a8dfde69fdfd admin admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:25:05.987317 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9ed5be-0241-40f2-8487-a8dfde69fdfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:25:05.988320 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9ed5be-0241-40f2-8487-a8dfde69fdfd admin admin] PATH_INFO: `/v3/roles` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:25:05.989045 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9ed5be-0241-40f2-8487-a8dfde69fdfd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:25:05.991106 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9ed5be-0241-40f2-8487-a8dfde69fdfd admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:25:06.005368 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1310/2642] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:25:05 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:25:06.006709 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0177bc-6b13-447e-9dcc-38cde7f2cd65 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:25:06.017046 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0177bc-6b13-447e-9dcc-38cde7f2cd65 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:25:06.022888 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71d8be1-2029-4611-9a96-0df580d30441 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:25:06.023785 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71d8be1-2029-4611-9a96-0df580d30441 admin admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:25:06.024067 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71d8be1-2029-4611-9a96-0df580d30441 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:25:06.024342 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71d8be1-2029-4611-9a96-0df580d30441 admin admin] PATH_INFO: `/v3/projects/9a1bd23dd3124af48a3f839ac0b30127/users/fc70a70d055c4531b532b418c91f5031/roles/cbf5c4c26898413d93e73b46b41046a3` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:25:06.040460 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71d8be1-2029-4611-9a96-0df580d30441 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a1bd23dd3124af48a3f839ac0b30127, user_id=fc70a70d055c4531b532b418c91f5031, role_id=cbf5c4c26898413d93e73b46b41046a3)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:25:06.043562 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71d8be1-2029-4611-9a96-0df580d30441 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:25:06.115023 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1333/2643] 10.222.0.23 () {60 vars in 1482 bytes} [Sat Jun 18 05:25:06 2022] PUT /identity/v3/projects/9a1bd23dd3124af48a3f839ac0b30127/users/fc70a70d055c4531b532b418c91f5031/roles/cbf5c4c26898413d93e73b46b41046a3 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:25:06.126906 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2896fb-679a-4403-a87f-10ae6cdc9b03 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:25:06.127202 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2896fb-679a-4403-a87f-10ae6cdc9b03 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:25:06.127487 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2896fb-679a-4403-a87f-10ae6cdc9b03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:25:06.154748 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-3c2896fb-679a-4403-a87f-10ae6cdc9b03 None None] MFA Rules not processed for user `fc70a70d055c4531b532b418c91f5031`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:25:06.156245 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-3c2896fb-679a-4403-a87f-10ae6cdc9b03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:25:06.222136 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1311/2644] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:25:06 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:25:06.227225 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71d8be1-2029-4611-9a96-0df580d30441 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:25:06.291177 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71d8be1-2029-4611-9a96-0df580d30441 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:25:06.296347 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739d889c-a1f9-4629-a24b-65243a0e4ae6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:25:06.297111 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739d889c-a1f9-4629-a24b-65243a0e4ae6 admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:25:06.297422 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739d889c-a1f9-4629-a24b-65243a0e4ae6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:25:06.297703 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739d889c-a1f9-4629-a24b-65243a0e4ae6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:25:06.298397 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739d889c-a1f9-4629-a24b-65243a0e4ae6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:25:06.300347 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739d889c-a1f9-4629-a24b-65243a0e4ae6 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:25:06.425746 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1334/2645] 10.222.0.23 () {60 vars in 1152 bytes} [Sat Jun 18 05:25:06 2022] POST /identity/v3/projects => generated 407 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:25:06.426959 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2896fb-679a-4403-a87f-10ae6cdc9b03 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:25:06.438632 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2896fb-679a-4403-a87f-10ae6cdc9b03 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:25:06.498373 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feadd3b4-61fd-4173-a349-f9718684addb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:25:06.499015 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feadd3b4-61fd-4173-a349-f9718684addb admin admin] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:25:06.499303 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feadd3b4-61fd-4173-a349-f9718684addb admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:25:06.499574 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feadd3b4-61fd-4173-a349-f9718684addb admin admin] PATH_INFO: `/v3/users` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:25:06.500250 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feadd3b4-61fd-4173-a349-f9718684addb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:25:06.502191 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feadd3b4-61fd-4173-a349-f9718684addb admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:25:06.607859 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1312/2646] 10.222.0.23 () {60 vars in 1143 bytes} [Sat Jun 18 05:25:06 2022] POST /identity/v3/users => generated 310 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:25:06.609010 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739d889c-a1f9-4629-a24b-65243a0e4ae6 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:25:06.621033 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739d889c-a1f9-4629-a24b-65243a0e4ae6 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:25:06.625940 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dbd0f9-a04d-4d28-a297-32e6e45a181f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:25:06.626614 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dbd0f9-a04d-4d28-a297-32e6e45a181f admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:25:06.626895 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dbd0f9-a04d-4d28-a297-32e6e45a181f admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:25:06.627172 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dbd0f9-a04d-4d28-a297-32e6e45a181f admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:25:06.627900 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dbd0f9-a04d-4d28-a297-32e6e45a181f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:25:06.629604 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dbd0f9-a04d-4d28-a297-32e6e45a181f admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:25:06.635519 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:25:06.641229 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1335/2647] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:25:06 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:25:06.645535 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feadd3b4-61fd-4173-a349-f9718684addb admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:25:06.656832 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feadd3b4-61fd-4173-a349-f9718684addb admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:25:06.661111 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bd0a12-1c81-4ee7-bccc-46107e4bb779 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:25:06.662857 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bd0a12-1c81-4ee7-bccc-46107e4bb779 admin admin] REQUEST_METHOD: `PUT` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:25:06.663138 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bd0a12-1c81-4ee7-bccc-46107e4bb779 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:25:06.663413 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bd0a12-1c81-4ee7-bccc-46107e4bb779 admin admin] PATH_INFO: `/v3/projects/1198fe7e01dc4189a7cbe042b27dd4e9/users/2de6431c589f425f87d43210a9f322fc/roles/21c65f2c51bd4dd283d13507d2714467` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:25:06.678581 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bd0a12-1c81-4ee7-bccc-46107e4bb779 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1198fe7e01dc4189a7cbe042b27dd4e9, user_id=2de6431c589f425f87d43210a9f322fc, role_id=21c65f2c51bd4dd283d13507d2714467)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:25:06.681230 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bd0a12-1c81-4ee7-bccc-46107e4bb779 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:25:06.741436 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1313/2648] 10.222.0.23 () {60 vars in 1482 bytes} [Sat Jun 18 05:25:06 2022] PUT /identity/v3/projects/1198fe7e01dc4189a7cbe042b27dd4e9/users/2de6431c589f425f87d43210a9f322fc/roles/21c65f2c51bd4dd283d13507d2714467 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 18 05:25:06.745318 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dbd0f9-a04d-4d28-a297-32e6e45a181f admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:25:06.755807 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dbd0f9-a04d-4d28-a297-32e6e45a181f admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:25:06.812067 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabd1769-2f96-4113-8d3d-0fbcb2a6cf34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:25:06.812796 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabd1769-2f96-4113-8d3d-0fbcb2a6cf34 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:25:06.813083 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabd1769-2f96-4113-8d3d-0fbcb2a6cf34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:25:06.813363 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabd1769-2f96-4113-8d3d-0fbcb2a6cf34 admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:25:06.814115 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabd1769-2f96-4113-8d3d-0fbcb2a6cf34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:25:06.815769 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabd1769-2f96-4113-8d3d-0fbcb2a6cf34 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:25:06.826229 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1336/2649] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:25:06 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:25:06.830319 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bd0a12-1c81-4ee7-bccc-46107e4bb779 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:25:06.841162 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bd0a12-1c81-4ee7-bccc-46107e4bb779 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:25:06.845802 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fd16a7-9bed-41b1-89c1-ac675570f7e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:25:06.846661 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fd16a7-9bed-41b1-89c1-ac675570f7e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:25:06.846943 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fd16a7-9bed-41b1-89c1-ac675570f7e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:25:06.847255 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fd16a7-9bed-41b1-89c1-ac675570f7e7 admin admin] PATH_INFO: `/v3/projects/1198fe7e01dc4189a7cbe042b27dd4e9/users/2de6431c589f425f87d43210a9f322fc/roles/21c65f2c51bd4dd283d13507d2714467` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:25:06.849330 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fd16a7-9bed-41b1-89c1-ac675570f7e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1198fe7e01dc4189a7cbe042b27dd4e9, user_id=2de6431c589f425f87d43210a9f322fc, role_id=21c65f2c51bd4dd283d13507d2714467)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:25:06.852080 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fd16a7-9bed-41b1-89c1-ac675570f7e7 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:25:06.862646 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1314/2650] 10.222.0.23 () {60 vars in 1482 bytes} [Sat Jun 18 05:25:06 2022] PUT /identity/v3/projects/1198fe7e01dc4189a7cbe042b27dd4e9/users/2de6431c589f425f87d43210a9f322fc/roles/21c65f2c51bd4dd283d13507d2714467 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:25:06.873084 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85fadf8-3159-4728-973f-7c0b4da3d4b6 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:25:06.873424 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85fadf8-3159-4728-973f-7c0b4da3d4b6 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:25:06.873712 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85fadf8-3159-4728-973f-7c0b4da3d4b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:25:06.900885 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-d85fadf8-3159-4728-973f-7c0b4da3d4b6 None None] MFA Rules not processed for user `2de6431c589f425f87d43210a9f322fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:25:06.902163 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-d85fadf8-3159-4728-973f-7c0b4da3d4b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:25:06.973279 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1337/2651] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:25:06 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:25:06.984220 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fd16a7-9bed-41b1-89c1-ac675570f7e7 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:25:06.986040 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-50fd16a7-9bed-41b1-89c1-ac675570f7e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:25:07.017526 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:25:07.044460 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fd16a7-9bed-41b1-89c1-ac675570f7e7 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:25:07.048171 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4612cefb-8a35-4853-bf6d-be1b06b6cdaf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9de0428abc8400aa338aba4f7c34c94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:25:07.048805 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4612cefb-8a35-4853-bf6d-be1b06b6cdaf service cinder] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:25:07.049085 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4612cefb-8a35-4853-bf6d-be1b06b6cdaf service cinder] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:25:07.049360 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4612cefb-8a35-4853-bf6d-be1b06b6cdaf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:25:07.059480 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4612cefb-8a35-4853-bf6d-be1b06b6cdaf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:25:07.064764 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4612cefb-8a35-4853-bf6d-be1b06b6cdaf service cinder] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:25:07.072423 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1315/2652] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:25:06 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:25:07.465031 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85fadf8-3159-4728-973f-7c0b4da3d4b6 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:25:07.478115 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85fadf8-3159-4728-973f-7c0b4da3d4b6 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:25:07.482651 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1669f340-2329-44e8-b18a-72abfbbb5cd3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9de0428abc8400aa338aba4f7c34c94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:25:07.483423 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1669f340-2329-44e8-b18a-72abfbbb5cd3 service cinder] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:25:07.483813 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1669f340-2329-44e8-b18a-72abfbbb5cd3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:25:07.484196 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1669f340-2329-44e8-b18a-72abfbbb5cd3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:25:07.542525 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1669f340-2329-44e8-b18a-72abfbbb5cd3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:25:07.543992 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1669f340-2329-44e8-b18a-72abfbbb5cd3 service cinder] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:25:07.553236 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1338/2653] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:25:07 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:25:17.203641 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=14 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:25:17.548866 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=91 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:25:19.000864 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e79a303-945b-427b-b64c-1621ab02933e None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:25:19.001301 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e79a303-945b-427b-b64c-1621ab02933e None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:25:19.001431 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e79a303-945b-427b-b64c-1621ab02933e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:25:19.025167 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-2e79a303-945b-427b-b64c-1621ab02933e None None] MFA Rules not processed for user `e5dc9999f6664c23855b66670f7d6a98`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:25:19.025167 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-2e79a303-945b-427b-b64c-1621ab02933e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:25:19.089060 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1316/2654] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:25:18 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:25:19.401733 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1669f340-2329-44e8-b18a-72abfbbb5cd3 service cinder] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:25:19.403171 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-1669f340-2329-44e8-b18a-72abfbbb5cd3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:25:19.461737 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1669f340-2329-44e8-b18a-72abfbbb5cd3 service cinder] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:25:19.465875 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef572b0-ee03-48f2-9568-a13c17be37e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c4d7cde8f4ad9aa479bb09e325db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:25:19.466748 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef572b0-ee03-48f2-9568-a13c17be37e7 service neutron] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:25:19.467058 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef572b0-ee03-48f2-9568-a13c17be37e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:25:19.467334 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef572b0-ee03-48f2-9568-a13c17be37e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:25:19.470605 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-eef572b0-ee03-48f2-9568-a13c17be37e7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:25:19.482142 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef572b0-ee03-48f2-9568-a13c17be37e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:25:19.487398 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef572b0-ee03-48f2-9568-a13c17be37e7 service neutron] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:25:19.492182 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1339/2655] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:25:19 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:25:19.635500 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e79a303-945b-427b-b64c-1621ab02933e None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:25:19.708785 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e79a303-945b-427b-b64c-1621ab02933e None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:25:19.714149 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37264d81-1a6b-4df5-bf6a-d8578f5322f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:25:19.714947 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37264d81-1a6b-4df5-bf6a-d8578f5322f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:25:19.715226 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37264d81-1a6b-4df5-bf6a-d8578f5322f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:25:19.715498 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37264d81-1a6b-4df5-bf6a-d8578f5322f1 admin admin] PATH_INFO: `/v3/users/fc70a70d055c4531b532b418c91f5031` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:25:19.716672 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37264d81-1a6b-4df5-bf6a-d8578f5322f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc70a70d055c4531b532b418c91f5031)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:25:19.718769 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37264d81-1a6b-4df5-bf6a-d8578f5322f1 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:25:19.934040 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-37264d81-1a6b-4df5-bf6a-d8578f5322f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc70a70d055c4531b532b418c91f5031'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:25:19.934733 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-37264d81-1a6b-4df5-bf6a-d8578f5322f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc70a70d055c4531b532b418c91f5031'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:25:20.004008 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-37264d81-1a6b-4df5-bf6a-d8578f5322f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc70a70d055c4531b532b418c91f5031'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:25:20.020703 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-37264d81-1a6b-4df5-bf6a-d8578f5322f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc70a70d055c4531b532b418c91f5031'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:25:20.026367 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-37264d81-1a6b-4df5-bf6a-d8578f5322f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc70a70d055c4531b532b418c91f5031'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:25:20.033271 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1317/2656] 10.222.0.23 () {58 vars in 1223 bytes} [Sat Jun 18 05:25:19 2022] DELETE /identity/v3/users/fc70a70d055c4531b532b418c91f5031 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:25:20.037613 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6956031e-7099-4f61-ad54-21f306d8e865 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:25:20.038078 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6956031e-7099-4f61-ad54-21f306d8e865 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:25:20.038544 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6956031e-7099-4f61-ad54-21f306d8e865 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:25:20.058331 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-6956031e-7099-4f61-ad54-21f306d8e865 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:25:20.061413 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-6956031e-7099-4f61-ad54-21f306d8e865 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:25:20.124105 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1340/2657] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:25:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:25:20.134515 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37264d81-1a6b-4df5-bf6a-d8578f5322f1 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:25:20.135927 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-37264d81-1a6b-4df5-bf6a-d8578f5322f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:25:20.164666 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:25:20.195974 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37264d81-1a6b-4df5-bf6a-d8578f5322f1 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:25:20.199717 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a00082-d675-4b3e-91df-da7e12b98c24 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c4d7cde8f4ad9aa479bb09e325db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:25:20.202068 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a00082-d675-4b3e-91df-da7e12b98c24 service neutron] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:25:20.202068 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a00082-d675-4b3e-91df-da7e12b98c24 service neutron] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:25:20.202068 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a00082-d675-4b3e-91df-da7e12b98c24 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:25:20.210681 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a00082-d675-4b3e-91df-da7e12b98c24 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:25:20.212099 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a00082-d675-4b3e-91df-da7e12b98c24 service neutron] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:25:20.222436 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1318/2658] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:25:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:25:20.737026 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6956031e-7099-4f61-ad54-21f306d8e865 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:25:20.738597 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-6956031e-7099-4f61-ad54-21f306d8e865 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:25:20.751090 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6956031e-7099-4f61-ad54-21f306d8e865 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:25:20.755789 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3388e492-9112-42ff-a2fe-8371c861ee98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:25:20.756807 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3388e492-9112-42ff-a2fe-8371c861ee98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:25:20.757713 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3388e492-9112-42ff-a2fe-8371c861ee98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:25:20.757713 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3388e492-9112-42ff-a2fe-8371c861ee98 admin admin] PATH_INFO: `/v3/projects/9a1bd23dd3124af48a3f839ac0b30127` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:25:20.758786 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3388e492-9112-42ff-a2fe-8371c861ee98 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a1bd23dd3124af48a3f839ac0b30127)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:25:20.760709 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3388e492-9112-42ff-a2fe-8371c861ee98 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:25:20.902118 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-3388e492-9112-42ff-a2fe-8371c861ee98 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a1bd23dd3124af48a3f839ac0b30127', 'request_id': 'req-3388e492-9112-42ff-a2fe-8371c861ee98'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:25:20.908029 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-3388e492-9112-42ff-a2fe-8371c861ee98 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a1bd23dd3124af48a3f839ac0b30127', 'request_id': 'req-3388e492-9112-42ff-a2fe-8371c861ee98'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:25:20.914883 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-3388e492-9112-42ff-a2fe-8371c861ee98 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a1bd23dd3124af48a3f839ac0b30127', 'request_id': 'req-3388e492-9112-42ff-a2fe-8371c861ee98'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:25:20.968790 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-3388e492-9112-42ff-a2fe-8371c861ee98 admin admin] The token cache is being invalidate because project 9a1bd23dd3124af48a3f839ac0b30127 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80469) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 18 05:25:20.969082 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-3388e492-9112-42ff-a2fe-8371c861ee98 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:25:20.969758 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-3388e492-9112-42ff-a2fe-8371c861ee98 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:25:20.971318 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1341/2659] 10.222.0.23 () {58 vars in 1232 bytes} [Sat Jun 18 05:25:20 2022] DELETE /identity/v3/projects/9a1bd23dd3124af48a3f839ac0b30127 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:25:20.977169 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a00082-d675-4b3e-91df-da7e12b98c24 service neutron] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:25:20.988533 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a00082-d675-4b3e-91df-da7e12b98c24 service neutron] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:25:21.048448 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e4e6b9-379f-4437-86b0-82d22a63bce8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:25:21.049320 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e4e6b9-379f-4437-86b0-82d22a63bce8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:25:21.049635 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e4e6b9-379f-4437-86b0-82d22a63bce8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:25:21.049917 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e4e6b9-379f-4437-86b0-82d22a63bce8 admin admin] PATH_INFO: `/v3/users/2de6431c589f425f87d43210a9f322fc` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:25:21.051123 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e4e6b9-379f-4437-86b0-82d22a63bce8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2de6431c589f425f87d43210a9f322fc)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:25:21.053033 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e4e6b9-379f-4437-86b0-82d22a63bce8 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:25:21.194048 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-63e4e6b9-379f-4437-86b0-82d22a63bce8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2de6431c589f425f87d43210a9f322fc'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:25:21.195284 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-63e4e6b9-379f-4437-86b0-82d22a63bce8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2de6431c589f425f87d43210a9f322fc'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:25:21.254967 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-63e4e6b9-379f-4437-86b0-82d22a63bce8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2de6431c589f425f87d43210a9f322fc'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:25:21.268986 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-63e4e6b9-379f-4437-86b0-82d22a63bce8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2de6431c589f425f87d43210a9f322fc'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:25:21.276812 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-63e4e6b9-379f-4437-86b0-82d22a63bce8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2de6431c589f425f87d43210a9f322fc'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:25:21.283242 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1319/2660] 10.222.0.23 () {58 vars in 1223 bytes} [Sat Jun 18 05:25:20 2022] DELETE /identity/v3/users/2de6431c589f425f87d43210a9f322fc => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:25:21.640111 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3388e492-9112-42ff-a2fe-8371c861ee98 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:25:21.646295 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-3388e492-9112-42ff-a2fe-8371c861ee98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:25:21.657432 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3388e492-9112-42ff-a2fe-8371c861ee98 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:25:21.700250 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:25:21.735212 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2442b243-a46b-4997-94d7-8b939ecaf1c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:25:21.736096 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2442b243-a46b-4997-94d7-8b939ecaf1c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:25:21.736382 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2442b243-a46b-4997-94d7-8b939ecaf1c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:25:21.736660 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2442b243-a46b-4997-94d7-8b939ecaf1c6 admin admin] PATH_INFO: `/v3/projects/1198fe7e01dc4189a7cbe042b27dd4e9` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:25:21.737877 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2442b243-a46b-4997-94d7-8b939ecaf1c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1198fe7e01dc4189a7cbe042b27dd4e9)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:25:21.739774 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2442b243-a46b-4997-94d7-8b939ecaf1c6 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:25:21.875717 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-2442b243-a46b-4997-94d7-8b939ecaf1c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1198fe7e01dc4189a7cbe042b27dd4e9', 'request_id': 'req-2442b243-a46b-4997-94d7-8b939ecaf1c6'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:25:21.881539 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-2442b243-a46b-4997-94d7-8b939ecaf1c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1198fe7e01dc4189a7cbe042b27dd4e9', 'request_id': 'req-2442b243-a46b-4997-94d7-8b939ecaf1c6'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:25:21.887829 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-2442b243-a46b-4997-94d7-8b939ecaf1c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1198fe7e01dc4189a7cbe042b27dd4e9', 'request_id': 'req-2442b243-a46b-4997-94d7-8b939ecaf1c6'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:25:21.956787 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-2442b243-a46b-4997-94d7-8b939ecaf1c6 admin admin] The token cache is being invalidate because project 1198fe7e01dc4189a7cbe042b27dd4e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80469) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 18 05:25:21.957076 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-2442b243-a46b-4997-94d7-8b939ecaf1c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:25:21.957710 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-2442b243-a46b-4997-94d7-8b939ecaf1c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:25:21.959269 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1342/2661] 10.222.0.23 () {58 vars in 1232 bytes} [Sat Jun 18 05:25:21 2022] DELETE /identity/v3/projects/1198fe7e01dc4189a7cbe042b27dd4e9 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:25:21.964763 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3653eb-c21f-4338-804e-0af810a8a4ec None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:25:21.965053 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3653eb-c21f-4338-804e-0af810a8a4ec None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:25:21.967009 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3653eb-c21f-4338-804e-0af810a8a4ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:25:21.984951 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-4b3653eb-c21f-4338-804e-0af810a8a4ec None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:25:21.986211 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-4b3653eb-c21f-4338-804e-0af810a8a4ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:25:22.030539 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:25:22.048764 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1320/2662] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:25:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:25:22.057245 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cef960-ee5a-48d1-be61-dbfc16eaa681 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:25:22.057567 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cef960-ee5a-48d1-be61-dbfc16eaa681 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:25:22.057852 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cef960-ee5a-48d1-be61-dbfc16eaa681 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:25:22.076740 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-72cef960-ee5a-48d1-be61-dbfc16eaa681 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:25:22.078400 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-72cef960-ee5a-48d1-be61-dbfc16eaa681 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:25:22.086691 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1343/2663] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:25:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:25:22.090890 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3653eb-c21f-4338-804e-0af810a8a4ec None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:25:22.101744 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3653eb-c21f-4338-804e-0af810a8a4ec None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:25:22.106853 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620f6984-2158-4f71-9f5b-9e4b1773250f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:25:22.107520 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620f6984-2158-4f71-9f5b-9e4b1773250f admin admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:25:22.107810 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620f6984-2158-4f71-9f5b-9e4b1773250f admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:25:22.108087 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620f6984-2158-4f71-9f5b-9e4b1773250f admin admin] PATH_INFO: `/v3/domains` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:25:22.108817 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620f6984-2158-4f71-9f5b-9e4b1773250f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 05:25:22.109131 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620f6984-2158-4f71-9f5b-9e4b1773250f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:25:22.110783 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620f6984-2158-4f71-9f5b-9e4b1773250f admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:25:22.123994 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1321/2664] 10.222.0.23 () {58 vars in 1152 bytes} [Sat Jun 18 05:25:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 05:25:22.128698 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cef960-ee5a-48d1-be61-dbfc16eaa681 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:25:22.138911 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cef960-ee5a-48d1-be61-dbfc16eaa681 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:25:22.143663 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73873911-00c8-4430-8e29-bcf9230a8110 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:25:22.144317 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73873911-00c8-4430-8e29-bcf9230a8110 admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:25:22.144599 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73873911-00c8-4430-8e29-bcf9230a8110 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:25:22.144902 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73873911-00c8-4430-8e29-bcf9230a8110 admin admin] PATH_INFO: `/v3/projects` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:25:22.145596 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73873911-00c8-4430-8e29-bcf9230a8110 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:25:22.148011 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73873911-00c8-4430-8e29-bcf9230a8110 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:25:22.239669 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620f6984-2158-4f71-9f5b-9e4b1773250f admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:25:22.241769 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-620f6984-2158-4f71-9f5b-9e4b1773250f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:25:22.246082 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1344/2665] 10.222.0.23 () {60 vars in 1152 bytes} [Sat Jun 18 05:25:22 2022] POST /identity/v3/projects => generated 365 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:25:22.250154 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73873911-00c8-4430-8e29-bcf9230a8110 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:25:22.315310 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620f6984-2158-4f71-9f5b-9e4b1773250f admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:25:22.324464 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73873911-00c8-4430-8e29-bcf9230a8110 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:25:22.334854 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46a6ba3-868b-4113-b72e-b16bfabd5abe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:25:22.340482 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46a6ba3-868b-4113-b72e-b16bfabd5abe admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:25:22.340482 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46a6ba3-868b-4113-b72e-b16bfabd5abe admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:25:22.340482 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46a6ba3-868b-4113-b72e-b16bfabd5abe admin admin] PATH_INFO: `/v3/users` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:25:22.340482 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46a6ba3-868b-4113-b72e-b16bfabd5abe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:25:22.340482 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46a6ba3-868b-4113-b72e-b16bfabd5abe admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:25:22.382360 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991d4a7c-5059-4353-a180-76fb635e368c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5323ffd91f864680b0c0084eb3f44d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:25:22.383176 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991d4a7c-5059-4353-a180-76fb635e368c service placement] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:25:22.383522 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991d4a7c-5059-4353-a180-76fb635e368c service placement] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:25:22.383796 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991d4a7c-5059-4353-a180-76fb635e368c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:25:22.386237 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-991d4a7c-5059-4353-a180-76fb635e368c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:25:22.399125 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991d4a7c-5059-4353-a180-76fb635e368c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:25:22.400763 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991d4a7c-5059-4353-a180-76fb635e368c service placement] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:25:22.411509 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1322/2666] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:25:22 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 18 05:25:22.422377 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1345/2667] 10.222.0.23 () {60 vars in 1143 bytes} [Sat Jun 18 05:25:22 2022] POST /identity/v3/users => generated 283 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 18 05:25:22.426883 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991d4a7c-5059-4353-a180-76fb635e368c service placement] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:25:22.439869 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991d4a7c-5059-4353-a180-76fb635e368c service placement] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:25:22.444756 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c586f0-a653-4ee6-b0dc-5b56ada236a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:25:22.445440 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c586f0-a653-4ee6-b0dc-5b56ada236a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:25:22.445736 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c586f0-a653-4ee6-b0dc-5b56ada236a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:25:22.446007 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c586f0-a653-4ee6-b0dc-5b56ada236a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:25:22.446725 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c586f0-a653-4ee6-b0dc-5b56ada236a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:25:22.448326 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c586f0-a653-4ee6-b0dc-5b56ada236a7 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:25:22.462588 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1323/2668] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:25:22 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:25:22.463765 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46a6ba3-868b-4113-b72e-b16bfabd5abe admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:25:22.474405 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46a6ba3-868b-4113-b72e-b16bfabd5abe admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:25:22.480607 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fdef49-f97f-4139-b0c7-be8913ec4072 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:25:22.481275 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fdef49-f97f-4139-b0c7-be8913ec4072 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:25:22.481587 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fdef49-f97f-4139-b0c7-be8913ec4072 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:25:22.481894 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fdef49-f97f-4139-b0c7-be8913ec4072 admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:25:22.483052 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fdef49-f97f-4139-b0c7-be8913ec4072 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:25:22.484548 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fdef49-f97f-4139-b0c7-be8913ec4072 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:25:22.497321 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1346/2669] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:25:22 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:25:22.501216 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c586f0-a653-4ee6-b0dc-5b56ada236a7 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:25:22.512383 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c586f0-a653-4ee6-b0dc-5b56ada236a7 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:25:22.517399 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b17169e-26ce-45b8-b594-418702fcf73e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:25:22.518381 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b17169e-26ce-45b8-b594-418702fcf73e admin admin] REQUEST_METHOD: `PUT` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:25:22.524102 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b17169e-26ce-45b8-b594-418702fcf73e admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:25:22.524102 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b17169e-26ce-45b8-b594-418702fcf73e admin admin] PATH_INFO: `/v3/projects/d2474edf9f754498b24131033ccbf275/users/e5f933096935482dbde2f354837d1059/roles/cbf5c4c26898413d93e73b46b41046a3` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:25:22.535040 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b17169e-26ce-45b8-b594-418702fcf73e admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2474edf9f754498b24131033ccbf275, user_id=e5f933096935482dbde2f354837d1059, role_id=cbf5c4c26898413d93e73b46b41046a3)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:25:22.537789 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b17169e-26ce-45b8-b594-418702fcf73e admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:25:22.608524 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1324/2670] 10.222.0.23 () {60 vars in 1482 bytes} [Sat Jun 18 05:25:22 2022] PUT /identity/v3/projects/d2474edf9f754498b24131033ccbf275/users/e5f933096935482dbde2f354837d1059/roles/cbf5c4c26898413d93e73b46b41046a3 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:25:22.619435 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebd5316-a3c1-4405-a5c8-4706f6c8253f None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:25:22.619730 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebd5316-a3c1-4405-a5c8-4706f6c8253f None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:25:22.620006 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebd5316-a3c1-4405-a5c8-4706f6c8253f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:25:22.633375 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:25:22.648879 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-4ebd5316-a3c1-4405-a5c8-4706f6c8253f None None] MFA Rules not processed for user `e5f933096935482dbde2f354837d1059`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:25:22.651727 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-4ebd5316-a3c1-4405-a5c8-4706f6c8253f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:25:22.717570 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1347/2671] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:25:22 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:25:22.722177 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b17169e-26ce-45b8-b594-418702fcf73e admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:25:22.756336 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:25:22.787091 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b17169e-26ce-45b8-b594-418702fcf73e admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:25:22.792102 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b21641-bca5-4fa8-98eb-b4ec754d0ec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:25:22.795524 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b21641-bca5-4fa8-98eb-b4ec754d0ec8 admin admin] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:25:22.795524 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b21641-bca5-4fa8-98eb-b4ec754d0ec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:25:22.795524 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b21641-bca5-4fa8-98eb-b4ec754d0ec8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:25:22.795524 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b21641-bca5-4fa8-98eb-b4ec754d0ec8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:25:22.796222 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b21641-bca5-4fa8-98eb-b4ec754d0ec8 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:25:22.895204 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1325/2672] 10.222.0.23 () {60 vars in 1152 bytes} [Sat Jun 18 05:25:22 2022] POST /identity/v3/projects => generated 367 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:25:22.896451 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebd5316-a3c1-4405-a5c8-4706f6c8253f None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:25:22.909206 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebd5316-a3c1-4405-a5c8-4706f6c8253f None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:25:22.968311 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177a2bb9-3951-44cd-afd4-f31239503c7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:25:22.968985 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177a2bb9-3951-44cd-afd4-f31239503c7c admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:25:22.969275 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177a2bb9-3951-44cd-afd4-f31239503c7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:25:22.969584 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177a2bb9-3951-44cd-afd4-f31239503c7c admin admin] PATH_INFO: `/v3/users` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:25:22.970274 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177a2bb9-3951-44cd-afd4-f31239503c7c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:25:22.972218 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177a2bb9-3951-44cd-afd4-f31239503c7c admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:25:23.026505 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1348/2673] 10.222.0.23 () {60 vars in 1143 bytes} [Sat Jun 18 05:25:22 2022] POST /identity/v3/users => generated 290 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:25:23.030921 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b21641-bca5-4fa8-98eb-b4ec754d0ec8 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:25:23.041710 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b21641-bca5-4fa8-98eb-b4ec754d0ec8 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:25:23.046652 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3da638-ed84-4d4a-87f5-f47283650780 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:25:23.047289 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3da638-ed84-4d4a-87f5-f47283650780 admin admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:25:23.047575 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3da638-ed84-4d4a-87f5-f47283650780 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:25:23.047848 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3da638-ed84-4d4a-87f5-f47283650780 admin admin] PATH_INFO: `/v3/roles` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:25:23.048577 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3da638-ed84-4d4a-87f5-f47283650780 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:25:23.050209 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3da638-ed84-4d4a-87f5-f47283650780 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:25:23.061589 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1326/2674] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:25:23 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:25:23.065772 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177a2bb9-3951-44cd-afd4-f31239503c7c admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:25:23.077287 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177a2bb9-3951-44cd-afd4-f31239503c7c admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:25:23.082249 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c22b97-34e4-4d38-a3b5-ba39924c974e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:25:23.083159 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c22b97-34e4-4d38-a3b5-ba39924c974e admin admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:25:23.083442 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c22b97-34e4-4d38-a3b5-ba39924c974e admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:25:23.083718 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c22b97-34e4-4d38-a3b5-ba39924c974e admin admin] PATH_INFO: `/v3/projects/ee7670eb12c542238bf61c716ab7c9f5/users/320f44ab491b4263a58f83b8fc1d8d29/roles/21c65f2c51bd4dd283d13507d2714467` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:25:23.099096 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c22b97-34e4-4d38-a3b5-ba39924c974e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee7670eb12c542238bf61c716ab7c9f5, user_id=320f44ab491b4263a58f83b8fc1d8d29, role_id=21c65f2c51bd4dd283d13507d2714467)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:25:23.101943 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c22b97-34e4-4d38-a3b5-ba39924c974e admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:25:23.244773 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1349/2675] 10.222.0.23 () {60 vars in 1482 bytes} [Sat Jun 18 05:25:23 2022] PUT /identity/v3/projects/ee7670eb12c542238bf61c716ab7c9f5/users/320f44ab491b4263a58f83b8fc1d8d29/roles/21c65f2c51bd4dd283d13507d2714467 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:25:23.248833 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3da638-ed84-4d4a-87f5-f47283650780 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:25:23.259563 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3da638-ed84-4d4a-87f5-f47283650780 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:25:23.318496 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063a5bd5-a639-47fc-921f-5cf5deaf4e97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:25:23.319160 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063a5bd5-a639-47fc-921f-5cf5deaf4e97 admin admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:25:23.320019 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063a5bd5-a639-47fc-921f-5cf5deaf4e97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:25:23.320607 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063a5bd5-a639-47fc-921f-5cf5deaf4e97 admin admin] PATH_INFO: `/v3/roles` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:25:23.321100 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063a5bd5-a639-47fc-921f-5cf5deaf4e97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:25:23.322749 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063a5bd5-a639-47fc-921f-5cf5deaf4e97 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:25:23.333631 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1327/2676] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:25:23 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:25:23.338378 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c22b97-34e4-4d38-a3b5-ba39924c974e admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:25:23.349171 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c22b97-34e4-4d38-a3b5-ba39924c974e admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:25:23.354291 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c02546-f6bb-47f0-af57-e8cb24fbe66b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:25:23.355170 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c02546-f6bb-47f0-af57-e8cb24fbe66b admin admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:25:23.355454 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c02546-f6bb-47f0-af57-e8cb24fbe66b admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:25:23.355769 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c02546-f6bb-47f0-af57-e8cb24fbe66b admin admin] PATH_INFO: `/v3/projects/ee7670eb12c542238bf61c716ab7c9f5/users/320f44ab491b4263a58f83b8fc1d8d29/roles/21c65f2c51bd4dd283d13507d2714467` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:25:23.357816 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c02546-f6bb-47f0-af57-e8cb24fbe66b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee7670eb12c542238bf61c716ab7c9f5, user_id=320f44ab491b4263a58f83b8fc1d8d29, role_id=21c65f2c51bd4dd283d13507d2714467)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:25:23.361076 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c02546-f6bb-47f0-af57-e8cb24fbe66b admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:25:23.382412 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1350/2677] 10.222.0.23 () {60 vars in 1482 bytes} [Sat Jun 18 05:25:23 2022] PUT /identity/v3/projects/ee7670eb12c542238bf61c716ab7c9f5/users/320f44ab491b4263a58f83b8fc1d8d29/roles/21c65f2c51bd4dd283d13507d2714467 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:25:23.384310 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dabdf1-56c6-4f94-b2d4-601a74f700d1 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:25:23.384954 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dabdf1-56c6-4f94-b2d4-601a74f700d1 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:25:23.385319 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dabdf1-56c6-4f94-b2d4-601a74f700d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:25:23.417539 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-b5dabdf1-56c6-4f94-b2d4-601a74f700d1 None None] MFA Rules not processed for user `320f44ab491b4263a58f83b8fc1d8d29`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:25:23.418810 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-b5dabdf1-56c6-4f94-b2d4-601a74f700d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:25:23.495365 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1328/2678] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:25:23 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 18 05:25:23.507715 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c02546-f6bb-47f0-af57-e8cb24fbe66b admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:25:23.509463 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-93c02546-f6bb-47f0-af57-e8cb24fbe66b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:25:23.564297 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:25:23.568154 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c02546-f6bb-47f0-af57-e8cb24fbe66b admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:25:23.571889 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe06e98c-311e-485c-a8d4-e6f06faed0cc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9de0428abc8400aa338aba4f7c34c94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:25:23.572535 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe06e98c-311e-485c-a8d4-e6f06faed0cc service cinder] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:25:23.572817 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe06e98c-311e-485c-a8d4-e6f06faed0cc service cinder] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:25:23.573094 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe06e98c-311e-485c-a8d4-e6f06faed0cc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:25:23.582115 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe06e98c-311e-485c-a8d4-e6f06faed0cc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:25:23.583555 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe06e98c-311e-485c-a8d4-e6f06faed0cc service cinder] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:25:23.592988 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1351/2679] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:25:23 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:25:23.834096 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dabdf1-56c6-4f94-b2d4-601a74f700d1 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:25:23.844470 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dabdf1-56c6-4f94-b2d4-601a74f700d1 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:25:23.848117 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f558dca3-a341-4794-afe2-1799debd14cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9de0428abc8400aa338aba4f7c34c94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:25:23.848764 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f558dca3-a341-4794-afe2-1799debd14cb service cinder] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:25:23.849052 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f558dca3-a341-4794-afe2-1799debd14cb service cinder] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:25:23.849333 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f558dca3-a341-4794-afe2-1799debd14cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:25:23.874618 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:25:23.908090 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f558dca3-a341-4794-afe2-1799debd14cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:25:23.909576 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f558dca3-a341-4794-afe2-1799debd14cb service cinder] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:25:23.919354 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1329/2680] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:25:23 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:25:30.240604 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe06e98c-311e-485c-a8d4-e6f06faed0cc service cinder] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:25:30.303027 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe06e98c-311e-485c-a8d4-e6f06faed0cc service cinder] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:25:30.307809 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c900aa1-0033-453e-a880-0d4e78a9ea84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:25:30.308670 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c900aa1-0033-453e-a880-0d4e78a9ea84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:25:30.308955 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c900aa1-0033-453e-a880-0d4e78a9ea84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:25:30.309255 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c900aa1-0033-453e-a880-0d4e78a9ea84 admin admin] PATH_INFO: `/v3/users/e5f933096935482dbde2f354837d1059` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:25:30.310470 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c900aa1-0033-453e-a880-0d4e78a9ea84 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e5f933096935482dbde2f354837d1059)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:25:30.312403 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c900aa1-0033-453e-a880-0d4e78a9ea84 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:25:30.463490 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-0c900aa1-0033-453e-a880-0d4e78a9ea84 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e5f933096935482dbde2f354837d1059'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:25:30.475299 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-0c900aa1-0033-453e-a880-0d4e78a9ea84 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e5f933096935482dbde2f354837d1059'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:25:30.476043 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-0c900aa1-0033-453e-a880-0d4e78a9ea84 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e5f933096935482dbde2f354837d1059'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:25:30.534950 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-0c900aa1-0033-453e-a880-0d4e78a9ea84 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e5f933096935482dbde2f354837d1059'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:25:30.543497 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-0c900aa1-0033-453e-a880-0d4e78a9ea84 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e5f933096935482dbde2f354837d1059'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:25:30.553181 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1352/2681] 10.222.0.23 () {58 vars in 1223 bytes} [Sat Jun 18 05:25:30 2022] DELETE /identity/v3/users/e5f933096935482dbde2f354837d1059 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:25:30.555206 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8170eefd-8fbb-4e59-abb6-407f96653c40 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:25:30.555649 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8170eefd-8fbb-4e59-abb6-407f96653c40 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:25:30.555933 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8170eefd-8fbb-4e59-abb6-407f96653c40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:25:30.573709 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-8170eefd-8fbb-4e59-abb6-407f96653c40 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:25:30.574928 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-8170eefd-8fbb-4e59-abb6-407f96653c40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:25:30.637911 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1330/2682] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:25:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 18 05:25:30.648177 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c900aa1-0033-453e-a880-0d4e78a9ea84 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:25:30.650045 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-0c900aa1-0033-453e-a880-0d4e78a9ea84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:25:30.710779 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c900aa1-0033-453e-a880-0d4e78a9ea84 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:25:30.714690 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340bcb20-5d56-46c1-9cf3-d4395ec48c7d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c4d7cde8f4ad9aa479bb09e325db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:25:30.715347 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340bcb20-5d56-46c1-9cf3-d4395ec48c7d service neutron] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:25:30.715649 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340bcb20-5d56-46c1-9cf3-d4395ec48c7d service neutron] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:25:30.723687 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340bcb20-5d56-46c1-9cf3-d4395ec48c7d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:25:30.725235 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340bcb20-5d56-46c1-9cf3-d4395ec48c7d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:25:30.727531 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340bcb20-5d56-46c1-9cf3-d4395ec48c7d service neutron] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:25:30.737024 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1353/2683] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:25:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:25:31.153919 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8170eefd-8fbb-4e59-abb6-407f96653c40 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:25:31.155470 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-8170eefd-8fbb-4e59-abb6-407f96653c40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:25:31.169313 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8170eefd-8fbb-4e59-abb6-407f96653c40 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:25:31.174078 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68971d14-631e-4519-ac36-4b46901ca3e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:25:31.174933 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68971d14-631e-4519-ac36-4b46901ca3e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:25:31.175218 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68971d14-631e-4519-ac36-4b46901ca3e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:25:31.175550 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68971d14-631e-4519-ac36-4b46901ca3e0 admin admin] PATH_INFO: `/v3/projects/d2474edf9f754498b24131033ccbf275` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:25:31.176717 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68971d14-631e-4519-ac36-4b46901ca3e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d2474edf9f754498b24131033ccbf275)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:25:31.179894 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68971d14-631e-4519-ac36-4b46901ca3e0 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:25:31.289925 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-68971d14-631e-4519-ac36-4b46901ca3e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd2474edf9f754498b24131033ccbf275', 'request_id': 'req-68971d14-631e-4519-ac36-4b46901ca3e0'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:25:31.295489 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-68971d14-631e-4519-ac36-4b46901ca3e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd2474edf9f754498b24131033ccbf275', 'request_id': 'req-68971d14-631e-4519-ac36-4b46901ca3e0'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:25:31.357706 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-68971d14-631e-4519-ac36-4b46901ca3e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd2474edf9f754498b24131033ccbf275', 'request_id': 'req-68971d14-631e-4519-ac36-4b46901ca3e0'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:25:31.370515 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-68971d14-631e-4519-ac36-4b46901ca3e0 admin admin] The token cache is being invalidate because project d2474edf9f754498b24131033ccbf275 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80468) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 18 05:25:31.370845 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-68971d14-631e-4519-ac36-4b46901ca3e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:25:31.371537 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-68971d14-631e-4519-ac36-4b46901ca3e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:25:31.373108 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1331/2684] 10.222.0.23 () {58 vars in 1232 bytes} [Sat Jun 18 05:25:31 2022] DELETE /identity/v3/projects/d2474edf9f754498b24131033ccbf275 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:25:31.379784 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340bcb20-5d56-46c1-9cf3-d4395ec48c7d service neutron] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:25:31.391119 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340bcb20-5d56-46c1-9cf3-d4395ec48c7d service neutron] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:25:31.400359 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:25:31.449991 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54207a1c-3766-4133-adab-2e073a1a05c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:25:31.450967 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54207a1c-3766-4133-adab-2e073a1a05c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:25:31.451692 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54207a1c-3766-4133-adab-2e073a1a05c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:25:31.452089 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54207a1c-3766-4133-adab-2e073a1a05c6 admin admin] PATH_INFO: `/v3/users/320f44ab491b4263a58f83b8fc1d8d29` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:25:31.453305 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54207a1c-3766-4133-adab-2e073a1a05c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=320f44ab491b4263a58f83b8fc1d8d29)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:25:31.455265 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54207a1c-3766-4133-adab-2e073a1a05c6 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:25:31.611991 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-54207a1c-3766-4133-adab-2e073a1a05c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '320f44ab491b4263a58f83b8fc1d8d29'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:25:31.625080 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-54207a1c-3766-4133-adab-2e073a1a05c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '320f44ab491b4263a58f83b8fc1d8d29'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:25:31.625863 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-54207a1c-3766-4133-adab-2e073a1a05c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '320f44ab491b4263a58f83b8fc1d8d29'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:25:31.700195 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-54207a1c-3766-4133-adab-2e073a1a05c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '320f44ab491b4263a58f83b8fc1d8d29'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:25:31.707354 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-54207a1c-3766-4133-adab-2e073a1a05c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '320f44ab491b4263a58f83b8fc1d8d29'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:25:31.713313 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1354/2685] 10.222.0.23 () {58 vars in 1223 bytes} [Sat Jun 18 05:25:31 2022] DELETE /identity/v3/users/320f44ab491b4263a58f83b8fc1d8d29 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 18 05:25:32.174355 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68971d14-631e-4519-ac36-4b46901ca3e0 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:25:32.175876 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-68971d14-631e-4519-ac36-4b46901ca3e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:25:32.199959 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54207a1c-3766-4133-adab-2e073a1a05c6 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:25:32.201687 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-54207a1c-3766-4133-adab-2e073a1a05c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:25:32.236065 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:25:32.239993 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68971d14-631e-4519-ac36-4b46901ca3e0 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:25:32.275966 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54207a1c-3766-4133-adab-2e073a1a05c6 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:25:32.288872 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4b11bb-84ea-4289-aa9f-a71c2a06a5a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:25:32.289824 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4b11bb-84ea-4289-aa9f-a71c2a06a5a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:25:32.290107 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4b11bb-84ea-4289-aa9f-a71c2a06a5a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:25:32.290380 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4b11bb-84ea-4289-aa9f-a71c2a06a5a5 admin admin] PATH_INFO: `/v3/projects/ee7670eb12c542238bf61c716ab7c9f5` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:25:32.290644 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0f67c3-c0fb-4b20-8f6c-db9a06249c74 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9de0428abc8400aa338aba4f7c34c94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:25:32.291343 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0f67c3-c0fb-4b20-8f6c-db9a06249c74 service cinder] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:25:32.291632 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4b11bb-84ea-4289-aa9f-a71c2a06a5a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee7670eb12c542238bf61c716ab7c9f5)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:25:32.291746 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0f67c3-c0fb-4b20-8f6c-db9a06249c74 service cinder] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:25:32.292008 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0f67c3-c0fb-4b20-8f6c-db9a06249c74 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:25:32.294480 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4b11bb-84ea-4289-aa9f-a71c2a06a5a5 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:25:32.295019 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-df0f67c3-c0fb-4b20-8f6c-db9a06249c74 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:25:32.354957 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0f67c3-c0fb-4b20-8f6c-db9a06249c74 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:25:32.356398 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0f67c3-c0fb-4b20-8f6c-db9a06249c74 service cinder] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:25:32.365570 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1332/2686] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:25:32 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:25:32.417788 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:25:32.427888 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-6c4b11bb-84ea-4289-aa9f-a71c2a06a5a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee7670eb12c542238bf61c716ab7c9f5', 'request_id': 'req-6c4b11bb-84ea-4289-aa9f-a71c2a06a5a5'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:25:32.433595 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-6c4b11bb-84ea-4289-aa9f-a71c2a06a5a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee7670eb12c542238bf61c716ab7c9f5', 'request_id': 'req-6c4b11bb-84ea-4289-aa9f-a71c2a06a5a5'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:25:32.439797 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-6c4b11bb-84ea-4289-aa9f-a71c2a06a5a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee7670eb12c542238bf61c716ab7c9f5', 'request_id': 'req-6c4b11bb-84ea-4289-aa9f-a71c2a06a5a5'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:25:32.506056 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-6c4b11bb-84ea-4289-aa9f-a71c2a06a5a5 admin admin] The token cache is being invalidate because project ee7670eb12c542238bf61c716ab7c9f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80469) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 18 05:25:32.506482 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-6c4b11bb-84ea-4289-aa9f-a71c2a06a5a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:25:32.508344 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-6c4b11bb-84ea-4289-aa9f-a71c2a06a5a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:25:32.509819 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1355/2687] 10.222.0.23 () {58 vars in 1232 bytes} [Sat Jun 18 05:25:32 2022] DELETE /identity/v3/projects/ee7670eb12c542238bf61c716ab7c9f5 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 18 05:25:32.514762 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c53df6b-5390-4564-90a1-5711c8de313d None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:25:32.515052 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c53df6b-5390-4564-90a1-5711c8de313d None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:25:32.515328 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c53df6b-5390-4564-90a1-5711c8de313d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:25:32.534229 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-4c53df6b-5390-4564-90a1-5711c8de313d None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:25:32.535414 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-4c53df6b-5390-4564-90a1-5711c8de313d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:25:32.605284 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1333/2688] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:25:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:25:32.607113 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078c7d42-c675-4e9f-8324-fcdd308e24be None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:25:32.607404 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078c7d42-c675-4e9f-8324-fcdd308e24be None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:25:32.607684 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078c7d42-c675-4e9f-8324-fcdd308e24be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:25:32.627819 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-078c7d42-c675-4e9f-8324-fcdd308e24be None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:25:32.629063 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-078c7d42-c675-4e9f-8324-fcdd308e24be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:25:32.637837 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1356/2689] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:25:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:25:32.642151 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c53df6b-5390-4564-90a1-5711c8de313d None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:25:32.652561 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c53df6b-5390-4564-90a1-5711c8de313d None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:25:32.657538 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dadc6b-aeb7-4aff-9ccf-f8e178b5d5aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:25:32.658194 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dadc6b-aeb7-4aff-9ccf-f8e178b5d5aa admin admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:25:32.658476 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dadc6b-aeb7-4aff-9ccf-f8e178b5d5aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:25:32.658751 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dadc6b-aeb7-4aff-9ccf-f8e178b5d5aa admin admin] PATH_INFO: `/v3/domains` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:25:32.659485 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17dadc6b-aeb7-4aff-9ccf-f8e178b5d5aa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 05:25:32.659797 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17dadc6b-aeb7-4aff-9ccf-f8e178b5d5aa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:25:32.661407 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17dadc6b-aeb7-4aff-9ccf-f8e178b5d5aa admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:25:32.674001 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1334/2690] 10.222.0.23 () {58 vars in 1152 bytes} [Sat Jun 18 05:25:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 18 05:25:32.678391 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078c7d42-c675-4e9f-8324-fcdd308e24be None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:25:32.687896 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078c7d42-c675-4e9f-8324-fcdd308e24be None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:25:32.693421 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c84ed11-58bd-4789-94a3-490ddbbde4e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:25:32.694090 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c84ed11-58bd-4789-94a3-490ddbbde4e0 admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:25:32.694366 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c84ed11-58bd-4789-94a3-490ddbbde4e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:25:32.694682 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c84ed11-58bd-4789-94a3-490ddbbde4e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:25:32.695398 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c84ed11-58bd-4789-94a3-490ddbbde4e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:25:32.697393 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c84ed11-58bd-4789-94a3-490ddbbde4e0 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:25:32.778752 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1357/2691] 10.222.0.23 () {60 vars in 1152 bytes} [Sat Jun 18 05:25:32 2022] POST /identity/v3/projects => generated 381 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:25:32.783499 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dadc6b-aeb7-4aff-9ccf-f8e178b5d5aa admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:25:32.794362 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dadc6b-aeb7-4aff-9ccf-f8e178b5d5aa admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:25:32.839265 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=100 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:25:32.851630 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c56be3-73cf-4845-8d8f-8327968d0f59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:25:32.852293 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c56be3-73cf-4845-8d8f-8327968d0f59 admin admin] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:25:32.852577 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c56be3-73cf-4845-8d8f-8327968d0f59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:25:32.852859 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c56be3-73cf-4845-8d8f-8327968d0f59 admin admin] PATH_INFO: `/v3/users` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:25:32.853615 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c56be3-73cf-4845-8d8f-8327968d0f59 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:25:32.855534 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c56be3-73cf-4845-8d8f-8327968d0f59 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:25:32.950461 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1335/2692] 10.222.0.23 () {60 vars in 1143 bytes} [Sat Jun 18 05:25:32 2022] POST /identity/v3/users => generated 291 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:25:32.951783 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c84ed11-58bd-4789-94a3-490ddbbde4e0 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:25:32.962838 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c84ed11-58bd-4789-94a3-490ddbbde4e0 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:25:32.967705 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2695a7d5-5cb6-4610-9dab-75937a1269eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:25:32.968444 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2695a7d5-5cb6-4610-9dab-75937a1269eb admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:25:32.968728 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2695a7d5-5cb6-4610-9dab-75937a1269eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:25:32.969001 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2695a7d5-5cb6-4610-9dab-75937a1269eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:25:32.969974 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2695a7d5-5cb6-4610-9dab-75937a1269eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:25:32.971657 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2695a7d5-5cb6-4610-9dab-75937a1269eb admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:25:32.982900 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1358/2693] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:25:32 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:25:32.987028 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c56be3-73cf-4845-8d8f-8327968d0f59 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:25:32.997980 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c56be3-73cf-4845-8d8f-8327968d0f59 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:25:33.002685 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9e63b5-0d43-4a3b-8ef0-6a6b29c8aa23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:25:33.003336 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9e63b5-0d43-4a3b-8ef0-6a6b29c8aa23 admin admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:25:33.009619 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9e63b5-0d43-4a3b-8ef0-6a6b29c8aa23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:25:33.009619 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9e63b5-0d43-4a3b-8ef0-6a6b29c8aa23 admin admin] PATH_INFO: `/v3/roles` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:25:33.009619 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9e63b5-0d43-4a3b-8ef0-6a6b29c8aa23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:25:33.009619 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9e63b5-0d43-4a3b-8ef0-6a6b29c8aa23 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:25:33.016541 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1336/2694] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:25:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:25:33.020967 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2695a7d5-5cb6-4610-9dab-75937a1269eb admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:25:33.032454 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2695a7d5-5cb6-4610-9dab-75937a1269eb admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:25:33.037232 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095f38c9-bc5b-4472-8916-63bf5476bff2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:25:33.038686 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095f38c9-bc5b-4472-8916-63bf5476bff2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:25:33.039134 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095f38c9-bc5b-4472-8916-63bf5476bff2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:25:33.039415 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095f38c9-bc5b-4472-8916-63bf5476bff2 admin admin] PATH_INFO: `/v3/projects/29e1f1ca503345d993d69e71729e1366/users/baba74c755ca4f3283c049b2e40609dc/roles/cbf5c4c26898413d93e73b46b41046a3` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:25:33.055180 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095f38c9-bc5b-4472-8916-63bf5476bff2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=29e1f1ca503345d993d69e71729e1366, user_id=baba74c755ca4f3283c049b2e40609dc, role_id=cbf5c4c26898413d93e73b46b41046a3)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:25:33.057890 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095f38c9-bc5b-4472-8916-63bf5476bff2 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:25:33.158047 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1359/2695] 10.222.0.23 () {60 vars in 1482 bytes} [Sat Jun 18 05:25:33 2022] PUT /identity/v3/projects/29e1f1ca503345d993d69e71729e1366/users/baba74c755ca4f3283c049b2e40609dc/roles/cbf5c4c26898413d93e73b46b41046a3 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 18 05:25:33.168462 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c742486-5814-47a5-ab4c-74835b5d420c None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:25:33.168751 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c742486-5814-47a5-ab4c-74835b5d420c None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:25:33.169028 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c742486-5814-47a5-ab4c-74835b5d420c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:25:33.195669 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-2c742486-5814-47a5-ab4c-74835b5d420c None None] MFA Rules not processed for user `baba74c755ca4f3283c049b2e40609dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:25:33.196864 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-2c742486-5814-47a5-ab4c-74835b5d420c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:25:33.260381 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1337/2696] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:25:33 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:25:33.264796 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095f38c9-bc5b-4472-8916-63bf5476bff2 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:25:33.327037 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095f38c9-bc5b-4472-8916-63bf5476bff2 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:25:33.331692 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b3d785-251a-424f-b7b9-db51fd07cd5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:25:33.332343 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b3d785-251a-424f-b7b9-db51fd07cd5c admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:25:33.332628 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b3d785-251a-424f-b7b9-db51fd07cd5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:25:33.332904 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b3d785-251a-424f-b7b9-db51fd07cd5c admin admin] PATH_INFO: `/v3/projects` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:25:33.333649 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b3d785-251a-424f-b7b9-db51fd07cd5c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:25:33.335903 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b3d785-251a-424f-b7b9-db51fd07cd5c admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:25:33.411240 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1360/2697] 10.222.0.23 () {60 vars in 1152 bytes} [Sat Jun 18 05:25:33 2022] POST /identity/v3/projects => generated 379 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:25:33.415566 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c742486-5814-47a5-ab4c-74835b5d420c None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:25:33.426223 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c742486-5814-47a5-ab4c-74835b5d420c None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:25:33.485942 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42e78c2-9a2d-4b92-a3ec-3f1f7459273e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:25:33.486627 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42e78c2-9a2d-4b92-a3ec-3f1f7459273e admin admin] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:25:33.486912 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42e78c2-9a2d-4b92-a3ec-3f1f7459273e admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:25:33.487189 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42e78c2-9a2d-4b92-a3ec-3f1f7459273e admin admin] PATH_INFO: `/v3/users` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:25:33.487884 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42e78c2-9a2d-4b92-a3ec-3f1f7459273e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:25:33.489890 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42e78c2-9a2d-4b92-a3ec-3f1f7459273e admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:25:33.532754 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2c4bae-5d6d-4534-bd7f-a9305fd750d8 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:25:33.533280 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2c4bae-5d6d-4534-bd7f-a9305fd750d8 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:25:33.533406 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2c4bae-5d6d-4534-bd7f-a9305fd750d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:25:33.551779 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-3b2c4bae-5d6d-4534-bd7f-a9305fd750d8 None None] MFA Rules not processed for user `f54c1aed6a784e40952d38784bf4ed6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:25:33.553810 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-3b2c4bae-5d6d-4534-bd7f-a9305fd750d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:25:33.594141 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1338/2698] 10.222.0.23 () {60 vars in 1143 bytes} [Sat Jun 18 05:25:33 2022] POST /identity/v3/users => generated 296 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:25:33.598504 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42e78c2-9a2d-4b92-a3ec-3f1f7459273e admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:25:33.599475 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:25:33.610072 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42e78c2-9a2d-4b92-a3ec-3f1f7459273e admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:25:33.616276 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b48cc9c-6b31-4de9-9fd3-2c803680b0cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:25:33.625295 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b48cc9c-6b31-4de9-9fd3-2c803680b0cc admin admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:25:33.625295 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b48cc9c-6b31-4de9-9fd3-2c803680b0cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:25:33.625295 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b48cc9c-6b31-4de9-9fd3-2c803680b0cc admin admin] PATH_INFO: `/v3/roles` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:25:33.625295 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b48cc9c-6b31-4de9-9fd3-2c803680b0cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:25:33.626033 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1361/2699] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:25:33 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 18 05:25:33.626132 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b48cc9c-6b31-4de9-9fd3-2c803680b0cc admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:25:33.631809 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1339/2700] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:25:33 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:25:33.635815 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2c4bae-5d6d-4534-bd7f-a9305fd750d8 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:25:33.650579 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2c4bae-5d6d-4534-bd7f-a9305fd750d8 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:25:33.657353 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3bc8c8-de11-48a7-88a2-907291dda2f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:25:33.657764 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3bc8c8-de11-48a7-88a2-907291dda2f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:25:33.657875 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3bc8c8-de11-48a7-88a2-907291dda2f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:25:33.658231 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3bc8c8-de11-48a7-88a2-907291dda2f9 admin admin] PATH_INFO: `/v3/projects/23ae02a69c364cb1b5345db5d8faee14/users/cd75255ae82441eea5140d9beaf2e5e6/roles/21c65f2c51bd4dd283d13507d2714467` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:25:33.675771 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3bc8c8-de11-48a7-88a2-907291dda2f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=23ae02a69c364cb1b5345db5d8faee14, user_id=cd75255ae82441eea5140d9beaf2e5e6, role_id=21c65f2c51bd4dd283d13507d2714467)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:25:33.679712 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3bc8c8-de11-48a7-88a2-907291dda2f9 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:25:33.974883 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1362/2701] 10.222.0.23 () {60 vars in 1482 bytes} [Sat Jun 18 05:25:33 2022] PUT /identity/v3/projects/23ae02a69c364cb1b5345db5d8faee14/users/cd75255ae82441eea5140d9beaf2e5e6/roles/21c65f2c51bd4dd283d13507d2714467 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:25:33.979718 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b48cc9c-6b31-4de9-9fd3-2c803680b0cc admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:25:33.989511 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b48cc9c-6b31-4de9-9fd3-2c803680b0cc admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:25:34.020549 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:25:34.050018 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00446154-2acd-4d9f-acef-b834fee67224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:25:34.050270 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00446154-2acd-4d9f-acef-b834fee67224 admin admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:25:34.050665 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00446154-2acd-4d9f-acef-b834fee67224 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:25:34.050812 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00446154-2acd-4d9f-acef-b834fee67224 admin admin] PATH_INFO: `/v3/roles` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:25:34.051531 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00446154-2acd-4d9f-acef-b834fee67224 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:25:34.053177 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00446154-2acd-4d9f-acef-b834fee67224 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:25:34.064059 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1340/2702] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:25:33 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:25:34.068267 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3bc8c8-de11-48a7-88a2-907291dda2f9 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:25:34.079756 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3bc8c8-de11-48a7-88a2-907291dda2f9 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:25:34.084260 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb174f0e-ee88-460a-b871-ef7d2f63ad7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:25:34.085926 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb174f0e-ee88-460a-b871-ef7d2f63ad7d admin admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:25:34.086414 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb174f0e-ee88-460a-b871-ef7d2f63ad7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:25:34.086850 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb174f0e-ee88-460a-b871-ef7d2f63ad7d admin admin] PATH_INFO: `/v3/projects/23ae02a69c364cb1b5345db5d8faee14/users/cd75255ae82441eea5140d9beaf2e5e6/roles/21c65f2c51bd4dd283d13507d2714467` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:25:34.088926 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb174f0e-ee88-460a-b871-ef7d2f63ad7d admin admin] RBAC: Authorizing `identity:create_grant(project_id=23ae02a69c364cb1b5345db5d8faee14, user_id=cd75255ae82441eea5140d9beaf2e5e6, role_id=21c65f2c51bd4dd283d13507d2714467)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:25:34.091618 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb174f0e-ee88-460a-b871-ef7d2f63ad7d admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:25:34.104189 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1363/2703] 10.222.0.23 () {60 vars in 1482 bytes} [Sat Jun 18 05:25:34 2022] PUT /identity/v3/projects/23ae02a69c364cb1b5345db5d8faee14/users/cd75255ae82441eea5140d9beaf2e5e6/roles/21c65f2c51bd4dd283d13507d2714467 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 18 05:25:34.114213 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9edd480-786d-4493-bfc5-0a711df104b3 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:25:34.114581 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9edd480-786d-4493-bfc5-0a711df104b3 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:25:34.114922 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9edd480-786d-4493-bfc5-0a711df104b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:25:34.145346 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-c9edd480-786d-4493-bfc5-0a711df104b3 None None] MFA Rules not processed for user `cd75255ae82441eea5140d9beaf2e5e6`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:25:34.146669 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-c9edd480-786d-4493-bfc5-0a711df104b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:25:34.222225 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1341/2704] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:25:34 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:25:34.229987 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb174f0e-ee88-460a-b871-ef7d2f63ad7d admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:25:34.236596 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-bb174f0e-ee88-460a-b871-ef7d2f63ad7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:25:34.329090 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb174f0e-ee88-460a-b871-ef7d2f63ad7d admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:25:34.329090 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e740de-cdc3-4d0f-af09-556eea79f269 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9de0428abc8400aa338aba4f7c34c94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:25:34.329090 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e740de-cdc3-4d0f-af09-556eea79f269 service cinder] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:25:34.329090 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e740de-cdc3-4d0f-af09-556eea79f269 service cinder] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:25:34.329544 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e740de-cdc3-4d0f-af09-556eea79f269 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:25:34.337276 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e740de-cdc3-4d0f-af09-556eea79f269 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:25:34.338965 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e740de-cdc3-4d0f-af09-556eea79f269 service cinder] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:25:34.353841 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1364/2705] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:25:34 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:25:34.863634 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9edd480-786d-4493-bfc5-0a711df104b3 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:25:34.927485 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9edd480-786d-4493-bfc5-0a711df104b3 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:25:34.932467 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de970748-39a2-4fe3-bb48-02eef820932f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:25:34.933314 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de970748-39a2-4fe3-bb48-02eef820932f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:25:34.933627 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de970748-39a2-4fe3-bb48-02eef820932f admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:25:34.933899 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de970748-39a2-4fe3-bb48-02eef820932f admin admin] PATH_INFO: `/v3/users/baba74c755ca4f3283c049b2e40609dc` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:25:34.935081 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de970748-39a2-4fe3-bb48-02eef820932f admin admin] RBAC: Authorizing `identity:delete_user(user_id=baba74c755ca4f3283c049b2e40609dc)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:25:34.937032 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de970748-39a2-4fe3-bb48-02eef820932f admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:25:35.073813 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-de970748-39a2-4fe3-bb48-02eef820932f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'baba74c755ca4f3283c049b2e40609dc'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:25:35.074398 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-de970748-39a2-4fe3-bb48-02eef820932f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'baba74c755ca4f3283c049b2e40609dc'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:25:35.123086 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-de970748-39a2-4fe3-bb48-02eef820932f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'baba74c755ca4f3283c049b2e40609dc'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:25:35.125090 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:25:35.135433 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-de970748-39a2-4fe3-bb48-02eef820932f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'baba74c755ca4f3283c049b2e40609dc'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:25:35.142177 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-de970748-39a2-4fe3-bb48-02eef820932f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'baba74c755ca4f3283c049b2e40609dc'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:25:35.148691 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1342/2706] 10.222.0.23 () {58 vars in 1223 bytes} [Sat Jun 18 05:25:34 2022] DELETE /identity/v3/users/baba74c755ca4f3283c049b2e40609dc => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:25:35.153148 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7e1301-1138-436f-b6c3-674ffb3baac2 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:25:35.153666 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7e1301-1138-436f-b6c3-674ffb3baac2 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:25:35.153819 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7e1301-1138-436f-b6c3-674ffb3baac2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:25:35.172185 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-4f7e1301-1138-436f-b6c3-674ffb3baac2 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:25:35.173405 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-4f7e1301-1138-436f-b6c3-674ffb3baac2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:25:35.232742 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1365/2707] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:25:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:25:35.242766 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de970748-39a2-4fe3-bb48-02eef820932f admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:25:35.244179 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-de970748-39a2-4fe3-bb48-02eef820932f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:25:35.303245 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de970748-39a2-4fe3-bb48-02eef820932f admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:25:35.307373 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5f9418-afae-4e61-95e7-d69d623063be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c4d7cde8f4ad9aa479bb09e325db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:25:35.308025 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5f9418-afae-4e61-95e7-d69d623063be service neutron] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:25:35.308312 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5f9418-afae-4e61-95e7-d69d623063be service neutron] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:25:35.308590 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5f9418-afae-4e61-95e7-d69d623063be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:25:35.318352 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5f9418-afae-4e61-95e7-d69d623063be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:25:35.320071 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5f9418-afae-4e61-95e7-d69d623063be service neutron] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:25:35.331605 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1343/2708] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:25:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:25:35.804512 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7e1301-1138-436f-b6c3-674ffb3baac2 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:25:35.805991 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-4f7e1301-1138-436f-b6c3-674ffb3baac2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:25:35.820038 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7e1301-1138-436f-b6c3-674ffb3baac2 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:25:35.825532 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c83256d-ed2d-4ab4-ad9d-a9d60e55addb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:25:35.826888 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c83256d-ed2d-4ab4-ad9d-a9d60e55addb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:25:35.827005 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c83256d-ed2d-4ab4-ad9d-a9d60e55addb admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:25:35.827215 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c83256d-ed2d-4ab4-ad9d-a9d60e55addb admin admin] PATH_INFO: `/v3/projects/29e1f1ca503345d993d69e71729e1366` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:25:35.828504 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c83256d-ed2d-4ab4-ad9d-a9d60e55addb admin admin] RBAC: Authorizing `identity:delete_project(project_id=29e1f1ca503345d993d69e71729e1366)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:25:35.830472 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c83256d-ed2d-4ab4-ad9d-a9d60e55addb admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:25:35.952194 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-2c83256d-ed2d-4ab4-ad9d-a9d60e55addb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '29e1f1ca503345d993d69e71729e1366', 'request_id': 'req-2c83256d-ed2d-4ab4-ad9d-a9d60e55addb'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:25:35.953372 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:25:35.958343 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-2c83256d-ed2d-4ab4-ad9d-a9d60e55addb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '29e1f1ca503345d993d69e71729e1366', 'request_id': 'req-2c83256d-ed2d-4ab4-ad9d-a9d60e55addb'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:25:35.965145 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-2c83256d-ed2d-4ab4-ad9d-a9d60e55addb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '29e1f1ca503345d993d69e71729e1366', 'request_id': 'req-2c83256d-ed2d-4ab4-ad9d-a9d60e55addb'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:25:36.019548 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-2c83256d-ed2d-4ab4-ad9d-a9d60e55addb admin admin] The token cache is being invalidate because project 29e1f1ca503345d993d69e71729e1366 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80469) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 18 05:25:36.019836 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-2c83256d-ed2d-4ab4-ad9d-a9d60e55addb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:25:36.020474 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-2c83256d-ed2d-4ab4-ad9d-a9d60e55addb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:25:36.022113 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1366/2709] 10.222.0.23 () {58 vars in 1232 bytes} [Sat Jun 18 05:25:35 2022] DELETE /identity/v3/projects/29e1f1ca503345d993d69e71729e1366 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:25:36.026811 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5f9418-afae-4e61-95e7-d69d623063be service neutron] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:25:36.036877 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5f9418-afae-4e61-95e7-d69d623063be service neutron] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:25:36.092194 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246ba46d-8029-40b1-870f-40d986d3b2f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:25:36.093055 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246ba46d-8029-40b1-870f-40d986d3b2f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:25:36.093345 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246ba46d-8029-40b1-870f-40d986d3b2f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:25:36.093649 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246ba46d-8029-40b1-870f-40d986d3b2f4 admin admin] PATH_INFO: `/v3/users/cd75255ae82441eea5140d9beaf2e5e6` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:25:36.094838 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246ba46d-8029-40b1-870f-40d986d3b2f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd75255ae82441eea5140d9beaf2e5e6)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:25:36.096749 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246ba46d-8029-40b1-870f-40d986d3b2f4 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:25:36.273904 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-246ba46d-8029-40b1-870f-40d986d3b2f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd75255ae82441eea5140d9beaf2e5e6'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:25:36.274583 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-246ba46d-8029-40b1-870f-40d986d3b2f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd75255ae82441eea5140d9beaf2e5e6'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:25:36.346143 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-246ba46d-8029-40b1-870f-40d986d3b2f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd75255ae82441eea5140d9beaf2e5e6'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:25:36.360254 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-246ba46d-8029-40b1-870f-40d986d3b2f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd75255ae82441eea5140d9beaf2e5e6'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:25:36.365664 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:25:36.370954 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-246ba46d-8029-40b1-870f-40d986d3b2f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd75255ae82441eea5140d9beaf2e5e6'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:25:36.374393 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1344/2710] 10.222.0.23 () {58 vars in 1223 bytes} [Sat Jun 18 05:25:36 2022] DELETE /identity/v3/users/cd75255ae82441eea5140d9beaf2e5e6 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:25:36.856299 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c83256d-ed2d-4ab4-ad9d-a9d60e55addb admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:25:36.858009 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-2c83256d-ed2d-4ab4-ad9d-a9d60e55addb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:25:36.871340 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c83256d-ed2d-4ab4-ad9d-a9d60e55addb admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:25:36.938132 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25f7aba-ec33-4b6f-94b7-4a7de9636f5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:25:36.938132 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25f7aba-ec33-4b6f-94b7-4a7de9636f5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:25:36.938132 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25f7aba-ec33-4b6f-94b7-4a7de9636f5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:25:36.938430 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25f7aba-ec33-4b6f-94b7-4a7de9636f5d admin admin] PATH_INFO: `/v3/projects/23ae02a69c364cb1b5345db5d8faee14` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:25:36.939583 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25f7aba-ec33-4b6f-94b7-4a7de9636f5d admin admin] RBAC: Authorizing `identity:delete_project(project_id=23ae02a69c364cb1b5345db5d8faee14)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:25:36.941495 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25f7aba-ec33-4b6f-94b7-4a7de9636f5d admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:25:37.076290 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-a25f7aba-ec33-4b6f-94b7-4a7de9636f5d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '23ae02a69c364cb1b5345db5d8faee14', 'request_id': 'req-a25f7aba-ec33-4b6f-94b7-4a7de9636f5d'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:25:37.082505 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-a25f7aba-ec33-4b6f-94b7-4a7de9636f5d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '23ae02a69c364cb1b5345db5d8faee14', 'request_id': 'req-a25f7aba-ec33-4b6f-94b7-4a7de9636f5d'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:25:37.089473 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-a25f7aba-ec33-4b6f-94b7-4a7de9636f5d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '23ae02a69c364cb1b5345db5d8faee14', 'request_id': 'req-a25f7aba-ec33-4b6f-94b7-4a7de9636f5d'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:25:37.203592 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-a25f7aba-ec33-4b6f-94b7-4a7de9636f5d admin admin] The token cache is being invalidate because project 23ae02a69c364cb1b5345db5d8faee14 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80469) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 18 05:25:37.203913 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-a25f7aba-ec33-4b6f-94b7-4a7de9636f5d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:25:37.204525 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-a25f7aba-ec33-4b6f-94b7-4a7de9636f5d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:25:37.206101 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1367/2711] 10.222.0.23 () {58 vars in 1232 bytes} [Sat Jun 18 05:25:36 2022] DELETE /identity/v3/projects/23ae02a69c364cb1b5345db5d8faee14 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:25:37.213916 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1574be81-f90f-40db-9ad5-ffd7859dabad None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:25:37.214314 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1574be81-f90f-40db-9ad5-ffd7859dabad None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:25:37.214804 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1574be81-f90f-40db-9ad5-ffd7859dabad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:25:37.240258 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-1574be81-f90f-40db-9ad5-ffd7859dabad None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:25:37.242985 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-1574be81-f90f-40db-9ad5-ffd7859dabad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:25:37.316463 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1345/2712] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:25:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:25:37.389617 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77752a7-503e-43d2-a23f-6387c4254e6d None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:25:37.389617 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77752a7-503e-43d2-a23f-6387c4254e6d None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:25:37.389617 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77752a7-503e-43d2-a23f-6387c4254e6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:25:37.411865 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-e77752a7-503e-43d2-a23f-6387c4254e6d None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:25:37.413837 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-e77752a7-503e-43d2-a23f-6387c4254e6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:25:37.422594 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1368/2713] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:25:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 18 05:25:37.426872 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1574be81-f90f-40db-9ad5-ffd7859dabad None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:25:37.437254 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1574be81-f90f-40db-9ad5-ffd7859dabad None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:25:37.442040 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de923011-5524-4d0b-8b11-b5a63437d80f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:25:37.442695 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de923011-5524-4d0b-8b11-b5a63437d80f admin admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:25:37.442985 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de923011-5524-4d0b-8b11-b5a63437d80f admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:25:37.443268 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de923011-5524-4d0b-8b11-b5a63437d80f admin admin] PATH_INFO: `/v3/domains` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:25:37.444014 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de923011-5524-4d0b-8b11-b5a63437d80f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 05:25:37.444340 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de923011-5524-4d0b-8b11-b5a63437d80f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:25:37.446009 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de923011-5524-4d0b-8b11-b5a63437d80f admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:25:37.457923 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1346/2714] 10.222.0.23 () {58 vars in 1152 bytes} [Sat Jun 18 05:25:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 05:25:37.463050 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77752a7-503e-43d2-a23f-6387c4254e6d None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:25:37.472612 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77752a7-503e-43d2-a23f-6387c4254e6d None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:25:37.478386 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2501e25-425e-4ea5-a3fd-ec24b30786a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:25:37.479039 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2501e25-425e-4ea5-a3fd-ec24b30786a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:25:37.479324 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2501e25-425e-4ea5-a3fd-ec24b30786a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:25:37.479606 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2501e25-425e-4ea5-a3fd-ec24b30786a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:25:37.480311 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2501e25-425e-4ea5-a3fd-ec24b30786a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:25:37.482375 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2501e25-425e-4ea5-a3fd-ec24b30786a3 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:25:37.600601 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1369/2715] 10.222.0.23 () {60 vars in 1152 bytes} [Sat Jun 18 05:25:37 2022] POST /identity/v3/projects => generated 381 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:25:37.605425 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de923011-5524-4d0b-8b11-b5a63437d80f admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:25:37.620154 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de923011-5524-4d0b-8b11-b5a63437d80f admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:25:37.691041 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200425fa-c4fb-445d-9bd2-694b529b5e9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:25:37.691932 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200425fa-c4fb-445d-9bd2-694b529b5e9f admin admin] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:25:37.692440 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200425fa-c4fb-445d-9bd2-694b529b5e9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:25:37.692805 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200425fa-c4fb-445d-9bd2-694b529b5e9f admin admin] PATH_INFO: `/v3/users` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:25:37.693631 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200425fa-c4fb-445d-9bd2-694b529b5e9f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:25:37.696637 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200425fa-c4fb-445d-9bd2-694b529b5e9f admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:25:37.794262 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1347/2716] 10.222.0.23 () {60 vars in 1143 bytes} [Sat Jun 18 05:25:37 2022] POST /identity/v3/users => generated 291 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:25:37.798172 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2501e25-425e-4ea5-a3fd-ec24b30786a3 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:25:37.810142 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2501e25-425e-4ea5-a3fd-ec24b30786a3 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:25:37.815326 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dd0a90-8da1-4637-95bf-cda9f8ba01bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:25:37.821146 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dd0a90-8da1-4637-95bf-cda9f8ba01bf admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:25:37.821146 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dd0a90-8da1-4637-95bf-cda9f8ba01bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:25:37.821146 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dd0a90-8da1-4637-95bf-cda9f8ba01bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:25:37.821146 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dd0a90-8da1-4637-95bf-cda9f8ba01bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:25:37.821146 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dd0a90-8da1-4637-95bf-cda9f8ba01bf admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:25:37.833104 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1370/2717] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:25:37 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:25:37.838059 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200425fa-c4fb-445d-9bd2-694b529b5e9f admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:25:37.850108 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200425fa-c4fb-445d-9bd2-694b529b5e9f admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:25:37.855973 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f55fda-1cfc-436a-8f72-363615537816 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:25:37.856810 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f55fda-1cfc-436a-8f72-363615537816 admin admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:25:37.857160 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f55fda-1cfc-436a-8f72-363615537816 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:25:37.857572 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f55fda-1cfc-436a-8f72-363615537816 admin admin] PATH_INFO: `/v3/roles` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:25:37.858374 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f55fda-1cfc-436a-8f72-363615537816 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:25:37.860066 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f55fda-1cfc-436a-8f72-363615537816 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:25:37.871669 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1348/2718] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:25:37 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:25:37.875581 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dd0a90-8da1-4637-95bf-cda9f8ba01bf admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:25:37.886311 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dd0a90-8da1-4637-95bf-cda9f8ba01bf admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:25:37.891241 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99a7788-67ab-4a3a-8581-4eed5d1cc040 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:25:37.892147 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99a7788-67ab-4a3a-8581-4eed5d1cc040 admin admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:25:37.898887 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99a7788-67ab-4a3a-8581-4eed5d1cc040 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:25:37.898887 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99a7788-67ab-4a3a-8581-4eed5d1cc040 admin admin] PATH_INFO: `/v3/projects/a6cbef87fbd54efeb4ead8baaa87eafa/users/8865fb5fed1444c38a215888130e4911/roles/cbf5c4c26898413d93e73b46b41046a3` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:25:37.909421 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99a7788-67ab-4a3a-8581-4eed5d1cc040 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6cbef87fbd54efeb4ead8baaa87eafa, user_id=8865fb5fed1444c38a215888130e4911, role_id=cbf5c4c26898413d93e73b46b41046a3)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:25:37.912151 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99a7788-67ab-4a3a-8581-4eed5d1cc040 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:25:37.998723 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1371/2719] 10.222.0.23 () {60 vars in 1482 bytes} [Sat Jun 18 05:25:37 2022] PUT /identity/v3/projects/a6cbef87fbd54efeb4ead8baaa87eafa/users/8865fb5fed1444c38a215888130e4911/roles/cbf5c4c26898413d93e73b46b41046a3 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:25:38.009282 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d4f282-8e3f-4667-bf48-a19b8a96e174 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:25:38.009625 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d4f282-8e3f-4667-bf48-a19b8a96e174 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:25:38.009942 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d4f282-8e3f-4667-bf48-a19b8a96e174 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:25:38.037842 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-b2d4f282-8e3f-4667-bf48-a19b8a96e174 None None] MFA Rules not processed for user `8865fb5fed1444c38a215888130e4911`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:25:38.039631 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-b2d4f282-8e3f-4667-bf48-a19b8a96e174 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:25:38.086989 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:25:38.105755 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1349/2720] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:25:38 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:25:38.110367 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99a7788-67ab-4a3a-8581-4eed5d1cc040 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:25:38.116057 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:25:38.193126 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99a7788-67ab-4a3a-8581-4eed5d1cc040 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:25:38.198299 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d935ad64-da76-4546-94ac-008b868f7a17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:25:38.198971 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d935ad64-da76-4546-94ac-008b868f7a17 admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:25:38.199262 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d935ad64-da76-4546-94ac-008b868f7a17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:25:38.199542 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d935ad64-da76-4546-94ac-008b868f7a17 admin admin] PATH_INFO: `/v3/projects` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:25:38.200252 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d935ad64-da76-4546-94ac-008b868f7a17 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:25:38.202332 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d935ad64-da76-4546-94ac-008b868f7a17 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:25:38.352790 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1372/2721] 10.222.0.23 () {60 vars in 1152 bytes} [Sat Jun 18 05:25:38 2022] POST /identity/v3/projects => generated 381 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:25:38.353986 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d4f282-8e3f-4667-bf48-a19b8a96e174 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:25:38.365090 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d4f282-8e3f-4667-bf48-a19b8a96e174 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:25:38.436615 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8145532d-9d62-461a-b219-24d8308f7d3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:25:38.437376 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8145532d-9d62-461a-b219-24d8308f7d3f admin admin] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:25:38.437773 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8145532d-9d62-461a-b219-24d8308f7d3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:25:38.438121 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8145532d-9d62-461a-b219-24d8308f7d3f admin admin] PATH_INFO: `/v3/users` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:25:38.438956 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8145532d-9d62-461a-b219-24d8308f7d3f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:25:38.441145 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8145532d-9d62-461a-b219-24d8308f7d3f admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:25:38.550249 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1350/2722] 10.222.0.23 () {60 vars in 1143 bytes} [Sat Jun 18 05:25:38 2022] POST /identity/v3/users => generated 297 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 18 05:25:38.554429 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d935ad64-da76-4546-94ac-008b868f7a17 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:25:38.565853 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d935ad64-da76-4546-94ac-008b868f7a17 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:25:38.571360 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fcaec5-fac7-414b-b08c-38266ff615e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:25:38.572133 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fcaec5-fac7-414b-b08c-38266ff615e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:25:38.572430 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fcaec5-fac7-414b-b08c-38266ff615e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:25:38.572685 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fcaec5-fac7-414b-b08c-38266ff615e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:25:38.573460 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fcaec5-fac7-414b-b08c-38266ff615e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:25:38.575082 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fcaec5-fac7-414b-b08c-38266ff615e7 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:25:38.586916 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1373/2723] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:25:38 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:25:38.591001 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8145532d-9d62-461a-b219-24d8308f7d3f admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:25:38.602256 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8145532d-9d62-461a-b219-24d8308f7d3f admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:25:38.607016 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8ea5c5-ee88-4f0a-bbea-6ec6578bed68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:25:38.607922 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8ea5c5-ee88-4f0a-bbea-6ec6578bed68 admin admin] REQUEST_METHOD: `PUT` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:25:38.608209 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8ea5c5-ee88-4f0a-bbea-6ec6578bed68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:25:38.608486 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8ea5c5-ee88-4f0a-bbea-6ec6578bed68 admin admin] PATH_INFO: `/v3/projects/22f7ba180dc64a32815df786f166d008/users/3a7f9fd3e06c4b5e8768d8687862a1ff/roles/21c65f2c51bd4dd283d13507d2714467` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:25:38.624077 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8ea5c5-ee88-4f0a-bbea-6ec6578bed68 admin admin] RBAC: Authorizing `identity:create_grant(project_id=22f7ba180dc64a32815df786f166d008, user_id=3a7f9fd3e06c4b5e8768d8687862a1ff, role_id=21c65f2c51bd4dd283d13507d2714467)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:25:38.626849 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8ea5c5-ee88-4f0a-bbea-6ec6578bed68 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:25:38.731189 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1351/2724] 10.222.0.23 () {60 vars in 1482 bytes} [Sat Jun 18 05:25:38 2022] PUT /identity/v3/projects/22f7ba180dc64a32815df786f166d008/users/3a7f9fd3e06c4b5e8768d8687862a1ff/roles/21c65f2c51bd4dd283d13507d2714467 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:25:38.735125 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fcaec5-fac7-414b-b08c-38266ff615e7 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:25:38.746342 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fcaec5-fac7-414b-b08c-38266ff615e7 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:25:38.832378 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252aa6f9-4761-4192-b89e-97a7d3269778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:25:38.832995 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252aa6f9-4761-4192-b89e-97a7d3269778 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:25:38.833328 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252aa6f9-4761-4192-b89e-97a7d3269778 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:25:38.833664 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252aa6f9-4761-4192-b89e-97a7d3269778 admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:25:38.835654 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252aa6f9-4761-4192-b89e-97a7d3269778 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:25:38.838131 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252aa6f9-4761-4192-b89e-97a7d3269778 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:25:38.850549 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1374/2725] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:25:38 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:25:38.855211 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8ea5c5-ee88-4f0a-bbea-6ec6578bed68 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:25:38.869234 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8ea5c5-ee88-4f0a-bbea-6ec6578bed68 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:25:38.876585 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e75d90-e504-4c27-b210-bf4017d401c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:25:38.877175 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e75d90-e504-4c27-b210-bf4017d401c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:25:38.877968 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e75d90-e504-4c27-b210-bf4017d401c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:25:38.877968 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e75d90-e504-4c27-b210-bf4017d401c5 admin admin] PATH_INFO: `/v3/projects/22f7ba180dc64a32815df786f166d008/users/3a7f9fd3e06c4b5e8768d8687862a1ff/roles/21c65f2c51bd4dd283d13507d2714467` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:25:38.880074 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e75d90-e504-4c27-b210-bf4017d401c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=22f7ba180dc64a32815df786f166d008, user_id=3a7f9fd3e06c4b5e8768d8687862a1ff, role_id=21c65f2c51bd4dd283d13507d2714467)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:25:38.883292 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e75d90-e504-4c27-b210-bf4017d401c5 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:25:38.901415 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1352/2726] 10.222.0.23 () {60 vars in 1482 bytes} [Sat Jun 18 05:25:38 2022] PUT /identity/v3/projects/22f7ba180dc64a32815df786f166d008/users/3a7f9fd3e06c4b5e8768d8687862a1ff/roles/21c65f2c51bd4dd283d13507d2714467 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 18 05:25:38.912056 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05507534-0edb-4e6f-b0e5-391d53acf905 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:25:38.912561 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05507534-0edb-4e6f-b0e5-391d53acf905 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:25:38.912696 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05507534-0edb-4e6f-b0e5-391d53acf905 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:25:38.942927 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-05507534-0edb-4e6f-b0e5-391d53acf905 None None] MFA Rules not processed for user `3a7f9fd3e06c4b5e8768d8687862a1ff`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:25:38.944149 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-05507534-0edb-4e6f-b0e5-391d53acf905 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:25:39.017837 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:25:39.022265 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1375/2727] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:25:38 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:25:39.034921 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e75d90-e504-4c27-b210-bf4017d401c5 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:25:39.036918 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-71e75d90-e504-4c27-b210-bf4017d401c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:25:39.102888 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e75d90-e504-4c27-b210-bf4017d401c5 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:25:39.106740 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db906926-3966-4b2d-893c-7c07278c4af4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9de0428abc8400aa338aba4f7c34c94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:25:39.107388 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db906926-3966-4b2d-893c-7c07278c4af4 service cinder] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:25:39.107674 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db906926-3966-4b2d-893c-7c07278c4af4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:25:39.108002 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db906926-3966-4b2d-893c-7c07278c4af4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:25:39.117694 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db906926-3966-4b2d-893c-7c07278c4af4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:25:39.119133 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db906926-3966-4b2d-893c-7c07278c4af4 service cinder] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:25:39.129252 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1353/2728] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:25:39 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:25:39.251932 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05507534-0edb-4e6f-b0e5-391d53acf905 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:25:39.262804 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05507534-0edb-4e6f-b0e5-391d53acf905 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:25:39.266406 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940c252c-98b0-4ce7-b0e5-a83871bfa42e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9de0428abc8400aa338aba4f7c34c94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:25:39.267080 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940c252c-98b0-4ce7-b0e5-a83871bfa42e service cinder] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:25:39.267360 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940c252c-98b0-4ce7-b0e5-a83871bfa42e service cinder] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:25:39.267636 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940c252c-98b0-4ce7-b0e5-a83871bfa42e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:25:39.327491 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940c252c-98b0-4ce7-b0e5-a83871bfa42e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:25:39.328978 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940c252c-98b0-4ce7-b0e5-a83871bfa42e service cinder] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:25:39.338109 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1376/2729] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:25:39 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:25:39.778940 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db906926-3966-4b2d-893c-7c07278c4af4 service cinder] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:25:39.832663 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:25:39.849168 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db906926-3966-4b2d-893c-7c07278c4af4 service cinder] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:25:39.860633 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ad4054-9d1a-48cc-9500-24d6a63a92e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:25:39.862091 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ad4054-9d1a-48cc-9500-24d6a63a92e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:25:39.862385 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ad4054-9d1a-48cc-9500-24d6a63a92e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:25:39.862726 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ad4054-9d1a-48cc-9500-24d6a63a92e0 admin admin] PATH_INFO: `/v3/users/8865fb5fed1444c38a215888130e4911` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:25:39.863976 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ad4054-9d1a-48cc-9500-24d6a63a92e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8865fb5fed1444c38a215888130e4911)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:25:39.866014 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ad4054-9d1a-48cc-9500-24d6a63a92e0 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:25:40.041305 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-d3ad4054-9d1a-48cc-9500-24d6a63a92e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8865fb5fed1444c38a215888130e4911'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:25:40.042048 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-d3ad4054-9d1a-48cc-9500-24d6a63a92e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8865fb5fed1444c38a215888130e4911'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:25:40.106082 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-d3ad4054-9d1a-48cc-9500-24d6a63a92e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8865fb5fed1444c38a215888130e4911'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:25:40.120857 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-d3ad4054-9d1a-48cc-9500-24d6a63a92e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8865fb5fed1444c38a215888130e4911'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:25:40.128431 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-d3ad4054-9d1a-48cc-9500-24d6a63a92e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8865fb5fed1444c38a215888130e4911'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:25:40.135341 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1354/2730] 10.222.0.23 () {58 vars in 1223 bytes} [Sat Jun 18 05:25:39 2022] DELETE /identity/v3/users/8865fb5fed1444c38a215888130e4911 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:25:40.140071 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc46a846-8686-49c8-8088-77e8d040afaa None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:25:40.140558 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc46a846-8686-49c8-8088-77e8d040afaa None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:25:40.140668 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc46a846-8686-49c8-8088-77e8d040afaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:25:40.159625 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-cc46a846-8686-49c8-8088-77e8d040afaa None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:25:40.160898 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-cc46a846-8686-49c8-8088-77e8d040afaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:25:40.230297 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1377/2731] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:25:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:25:40.244156 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ad4054-9d1a-48cc-9500-24d6a63a92e0 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:25:40.244156 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-d3ad4054-9d1a-48cc-9500-24d6a63a92e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:25:40.304810 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ad4054-9d1a-48cc-9500-24d6a63a92e0 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:25:40.308552 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2ecd4f-9929-40e2-b99f-e5c7cc8e4bf9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c4d7cde8f4ad9aa479bb09e325db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:25:40.309194 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2ecd4f-9929-40e2-b99f-e5c7cc8e4bf9 service neutron] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:25:40.309553 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2ecd4f-9929-40e2-b99f-e5c7cc8e4bf9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:25:40.309840 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2ecd4f-9929-40e2-b99f-e5c7cc8e4bf9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:25:40.320411 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2ecd4f-9929-40e2-b99f-e5c7cc8e4bf9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:25:40.325802 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2ecd4f-9929-40e2-b99f-e5c7cc8e4bf9 service neutron] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:25:40.331663 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1355/2732] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:25:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:25:40.781752 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc46a846-8686-49c8-8088-77e8d040afaa None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:25:40.781752 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-cc46a846-8686-49c8-8088-77e8d040afaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:25:40.792607 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc46a846-8686-49c8-8088-77e8d040afaa None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:25:40.797927 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cbe5fd-0cae-4340-accf-eac9cbf42d19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:25:40.799659 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cbe5fd-0cae-4340-accf-eac9cbf42d19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:25:40.799816 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cbe5fd-0cae-4340-accf-eac9cbf42d19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:25:40.800238 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cbe5fd-0cae-4340-accf-eac9cbf42d19 admin admin] PATH_INFO: `/v3/projects/a6cbef87fbd54efeb4ead8baaa87eafa` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:25:40.802501 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cbe5fd-0cae-4340-accf-eac9cbf42d19 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a6cbef87fbd54efeb4ead8baaa87eafa)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:25:40.808382 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cbe5fd-0cae-4340-accf-eac9cbf42d19 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:25:40.935005 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-c3cbe5fd-0cae-4340-accf-eac9cbf42d19 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6cbef87fbd54efeb4ead8baaa87eafa', 'request_id': 'req-c3cbe5fd-0cae-4340-accf-eac9cbf42d19'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:25:40.945631 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-c3cbe5fd-0cae-4340-accf-eac9cbf42d19 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6cbef87fbd54efeb4ead8baaa87eafa', 'request_id': 'req-c3cbe5fd-0cae-4340-accf-eac9cbf42d19'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:25:40.947823 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-c3cbe5fd-0cae-4340-accf-eac9cbf42d19 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6cbef87fbd54efeb4ead8baaa87eafa', 'request_id': 'req-c3cbe5fd-0cae-4340-accf-eac9cbf42d19'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:25:40.968728 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0482c869-2f08-4bae-9b24-9509bd5423af None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:25:40.971823 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0482c869-2f08-4bae-9b24-9509bd5423af None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:25:40.971823 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0482c869-2f08-4bae-9b24-9509bd5423af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:25:40.986677 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-0482c869-2f08-4bae-9b24-9509bd5423af None None] MFA Rules not processed for user `6d24c8fbdc7246e2af87004f2985a9f5`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:25:40.987877 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-0482c869-2f08-4bae-9b24-9509bd5423af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:25:41.025730 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-c3cbe5fd-0cae-4340-accf-eac9cbf42d19 admin admin] The token cache is being invalidate because project a6cbef87fbd54efeb4ead8baaa87eafa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80469) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 18 05:25:41.026533 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-c3cbe5fd-0cae-4340-accf-eac9cbf42d19 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:25:41.026973 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-c3cbe5fd-0cae-4340-accf-eac9cbf42d19 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:25:41.028534 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1378/2733] 10.222.0.23 () {58 vars in 1232 bytes} [Sat Jun 18 05:25:40 2022] DELETE /identity/v3/projects/a6cbef87fbd54efeb4ead8baaa87eafa => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:25:41.032818 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cbe5fd-0cae-4340-accf-eac9cbf42d19 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:25:41.034189 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-c3cbe5fd-0cae-4340-accf-eac9cbf42d19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:25:41.044537 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1356/2734] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:25:40 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 18 05:25:41.061781 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:25:41.102650 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cbe5fd-0cae-4340-accf-eac9cbf42d19 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:25:41.106652 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb353010-610d-4194-9655-ababbe92cac0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:25:41.107590 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb353010-610d-4194-9655-ababbe92cac0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:25:41.107898 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb353010-610d-4194-9655-ababbe92cac0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:25:41.108199 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb353010-610d-4194-9655-ababbe92cac0 admin admin] PATH_INFO: `/v3/users/3a7f9fd3e06c4b5e8768d8687862a1ff` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:25:41.109375 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb353010-610d-4194-9655-ababbe92cac0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a7f9fd3e06c4b5e8768d8687862a1ff)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:25:41.111378 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb353010-610d-4194-9655-ababbe92cac0 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:25:41.361153 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-eb353010-610d-4194-9655-ababbe92cac0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a7f9fd3e06c4b5e8768d8687862a1ff'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:25:41.374550 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-eb353010-610d-4194-9655-ababbe92cac0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a7f9fd3e06c4b5e8768d8687862a1ff'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:25:41.375279 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-eb353010-610d-4194-9655-ababbe92cac0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a7f9fd3e06c4b5e8768d8687862a1ff'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:25:41.455144 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-eb353010-610d-4194-9655-ababbe92cac0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a7f9fd3e06c4b5e8768d8687862a1ff'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:25:41.465587 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-eb353010-610d-4194-9655-ababbe92cac0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a7f9fd3e06c4b5e8768d8687862a1ff'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:25:41.473064 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1379/2735] 10.222.0.23 () {58 vars in 1223 bytes} [Sat Jun 18 05:25:41 2022] DELETE /identity/v3/users/3a7f9fd3e06c4b5e8768d8687862a1ff => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 18 05:25:42.062724 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0482c869-2f08-4bae-9b24-9509bd5423af None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:25:42.077927 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0482c869-2f08-4bae-9b24-9509bd5423af None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:25:42.089091 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:25:42.140575 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f0d725-62e5-4e5d-b103-476230d4cf75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:25:42.145034 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f0d725-62e5-4e5d-b103-476230d4cf75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:25:42.145034 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f0d725-62e5-4e5d-b103-476230d4cf75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:25:42.145034 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f0d725-62e5-4e5d-b103-476230d4cf75 admin admin] PATH_INFO: `/v3/projects/22f7ba180dc64a32815df786f166d008` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:25:42.145034 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f0d725-62e5-4e5d-b103-476230d4cf75 admin admin] RBAC: Authorizing `identity:delete_project(project_id=22f7ba180dc64a32815df786f166d008)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:25:42.145430 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f0d725-62e5-4e5d-b103-476230d4cf75 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:25:42.310694 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-38f0d725-62e5-4e5d-b103-476230d4cf75 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '22f7ba180dc64a32815df786f166d008', 'request_id': 'req-38f0d725-62e5-4e5d-b103-476230d4cf75'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:25:42.316229 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-38f0d725-62e5-4e5d-b103-476230d4cf75 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '22f7ba180dc64a32815df786f166d008', 'request_id': 'req-38f0d725-62e5-4e5d-b103-476230d4cf75'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:25:42.385847 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-38f0d725-62e5-4e5d-b103-476230d4cf75 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '22f7ba180dc64a32815df786f166d008', 'request_id': 'req-38f0d725-62e5-4e5d-b103-476230d4cf75'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:25:42.402957 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-38f0d725-62e5-4e5d-b103-476230d4cf75 admin admin] The token cache is being invalidate because project 22f7ba180dc64a32815df786f166d008 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80468) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 18 05:25:42.402957 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-38f0d725-62e5-4e5d-b103-476230d4cf75 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:25:42.409057 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-38f0d725-62e5-4e5d-b103-476230d4cf75 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:25:42.410753 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1357/2736] 10.222.0.23 () {58 vars in 1232 bytes} [Sat Jun 18 05:25:42 2022] DELETE /identity/v3/projects/22f7ba180dc64a32815df786f166d008 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:25:51.469113 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=43,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:25:52.547295 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=45,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:26:31.518041 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb353010-610d-4194-9655-ababbe92cac0 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:26:31.519458 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-eb353010-610d-4194-9655-ababbe92cac0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:26:31.576606 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb353010-610d-4194-9655-ababbe92cac0 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:26:31.624152 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416e115e-1c50-4bcf-94dd-dd5d5545afa9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c4d7cde8f4ad9aa479bb09e325db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:26:31.624796 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416e115e-1c50-4bcf-94dd-dd5d5545afa9 service neutron] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:26:31.625080 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416e115e-1c50-4bcf-94dd-dd5d5545afa9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:26:31.625355 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416e115e-1c50-4bcf-94dd-dd5d5545afa9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:26:31.627678 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-416e115e-1c50-4bcf-94dd-dd5d5545afa9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:26:31.695703 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416e115e-1c50-4bcf-94dd-dd5d5545afa9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:26:31.697132 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416e115e-1c50-4bcf-94dd-dd5d5545afa9 service neutron] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:26:31.708263 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1380/2737] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:26:31 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:26:34.737354 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f0d725-62e5-4e5d-b103-476230d4cf75 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:26:34.738805 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-38f0d725-62e5-4e5d-b103-476230d4cf75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:26:34.797576 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f0d725-62e5-4e5d-b103-476230d4cf75 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:26:34.801877 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5d3e70-8552-4593-8132-64bbfe34df83 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9de0428abc8400aa338aba4f7c34c94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:26:34.802570 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5d3e70-8552-4593-8132-64bbfe34df83 service cinder] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:26:34.802858 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5d3e70-8552-4593-8132-64bbfe34df83 service cinder] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:26:34.803138 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5d3e70-8552-4593-8132-64bbfe34df83 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:26:34.804906 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-0a5d3e70-8552-4593-8132-64bbfe34df83 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:26:34.818425 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5d3e70-8552-4593-8132-64bbfe34df83 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:26:34.819818 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5d3e70-8552-4593-8132-64bbfe34df83 service cinder] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:26:34.831753 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1358/2738] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:26:34 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:26:41.703165 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=74 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:26:44.049693 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416e115e-1c50-4bcf-94dd-dd5d5545afa9 service neutron] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:26:44.052062 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-416e115e-1c50-4bcf-94dd-dd5d5545afa9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:26:44.123058 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416e115e-1c50-4bcf-94dd-dd5d5545afa9 service neutron] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:26:44.128616 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176e8b89-1061-4d68-9831-6177c3493441 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5323ffd91f864680b0c0084eb3f44d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:26:44.130086 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176e8b89-1061-4d68-9831-6177c3493441 service placement] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:26:44.130507 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176e8b89-1061-4d68-9831-6177c3493441 service placement] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:26:44.130810 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176e8b89-1061-4d68-9831-6177c3493441 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:26:44.132621 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-176e8b89-1061-4d68-9831-6177c3493441 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:26:44.145141 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176e8b89-1061-4d68-9831-6177c3493441 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:26:44.146825 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176e8b89-1061-4d68-9831-6177c3493441 service placement] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:26:44.158994 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1381/2739] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:26:44 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:26:44.159431 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5d3e70-8552-4593-8132-64bbfe34df83 service cinder] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:26:44.161256 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-0a5d3e70-8552-4593-8132-64bbfe34df83 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:26:44.228742 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5d3e70-8552-4593-8132-64bbfe34df83 service cinder] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:26:44.233662 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fcce44-3425-408d-abe8-dbfccf77078c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:26:44.234535 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fcce44-3425-408d-abe8-dbfccf77078c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:26:44.234870 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fcce44-3425-408d-abe8-dbfccf77078c admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:26:44.235094 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fcce44-3425-408d-abe8-dbfccf77078c admin admin] PATH_INFO: `/v3/users/f54c1aed6a784e40952d38784bf4ed6b` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:26:44.236306 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fcce44-3425-408d-abe8-dbfccf77078c admin admin] RBAC: Authorizing `identity:delete_user(user_id=f54c1aed6a784e40952d38784bf4ed6b)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:26:44.238872 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fcce44-3425-408d-abe8-dbfccf77078c admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:26:44.450618 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-30fcce44-3425-408d-abe8-dbfccf77078c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f54c1aed6a784e40952d38784bf4ed6b'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:26:44.451289 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-30fcce44-3425-408d-abe8-dbfccf77078c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f54c1aed6a784e40952d38784bf4ed6b'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:26:44.516569 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-30fcce44-3425-408d-abe8-dbfccf77078c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f54c1aed6a784e40952d38784bf4ed6b'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:26:44.530514 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-30fcce44-3425-408d-abe8-dbfccf77078c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f54c1aed6a784e40952d38784bf4ed6b'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:26:44.538907 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-30fcce44-3425-408d-abe8-dbfccf77078c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f54c1aed6a784e40952d38784bf4ed6b'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:26:44.545109 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1359/2740] 10.222.0.23 () {58 vars in 1223 bytes} [Sat Jun 18 05:26:44 2022] DELETE /identity/v3/users/f54c1aed6a784e40952d38784bf4ed6b => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 18 05:26:44.838628 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176e8b89-1061-4d68-9831-6177c3493441 service placement] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:26:44.850838 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176e8b89-1061-4d68-9831-6177c3493441 service placement] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:26:44.921683 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81a3659-63d3-420d-8173-8bddc463c09b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:26:44.922585 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81a3659-63d3-420d-8173-8bddc463c09b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:26:44.922871 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81a3659-63d3-420d-8173-8bddc463c09b admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:26:44.923180 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81a3659-63d3-420d-8173-8bddc463c09b admin admin] PATH_INFO: `/v3/projects/d8863ced0efc4b95806761cc2d8a6f78` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:26:44.924399 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81a3659-63d3-420d-8173-8bddc463c09b admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8863ced0efc4b95806761cc2d8a6f78)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:26:44.926445 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81a3659-63d3-420d-8173-8bddc463c09b admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:26:45.061027 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-f81a3659-63d3-420d-8173-8bddc463c09b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8863ced0efc4b95806761cc2d8a6f78', 'request_id': 'req-f81a3659-63d3-420d-8173-8bddc463c09b'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:26:45.067692 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-f81a3659-63d3-420d-8173-8bddc463c09b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8863ced0efc4b95806761cc2d8a6f78', 'request_id': 'req-f81a3659-63d3-420d-8173-8bddc463c09b'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:26:45.073977 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-f81a3659-63d3-420d-8173-8bddc463c09b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8863ced0efc4b95806761cc2d8a6f78', 'request_id': 'req-f81a3659-63d3-420d-8173-8bddc463c09b'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:26:45.173054 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-f81a3659-63d3-420d-8173-8bddc463c09b admin admin] The token cache is being invalidate because project d8863ced0efc4b95806761cc2d8a6f78 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80469) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 18 05:26:45.173341 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-f81a3659-63d3-420d-8173-8bddc463c09b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:26:45.174026 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-f81a3659-63d3-420d-8173-8bddc463c09b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:26:45.175595 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1382/2741] 10.222.0.23 () {58 vars in 1232 bytes} [Sat Jun 18 05:26:44 2022] DELETE /identity/v3/projects/d8863ced0efc4b95806761cc2d8a6f78 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:26:45.405683 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e1e903-bdbf-4519-9573-3f17fa517d55 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:26:45.406460 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e1e903-bdbf-4519-9573-3f17fa517d55 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:26:45.406846 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e1e903-bdbf-4519-9573-3f17fa517d55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:26:45.426515 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-33e1e903-bdbf-4519-9573-3f17fa517d55 None None] MFA Rules not processed for user `e5dc9999f6664c23855b66670f7d6a98`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:26:45.427936 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-33e1e903-bdbf-4519-9573-3f17fa517d55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:26:45.445092 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:26:45.484934 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1360/2742] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:26:45 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:26:54.410709 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81a3659-63d3-420d-8173-8bddc463c09b admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:26:54.412293 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-f81a3659-63d3-420d-8173-8bddc463c09b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:26:54.436909 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:26:54.473218 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81a3659-63d3-420d-8173-8bddc463c09b admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:26:54.477130 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4a7488-bfba-48b6-aaca-67f3b711bf63 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c4d7cde8f4ad9aa479bb09e325db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:26:54.477834 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4a7488-bfba-48b6-aaca-67f3b711bf63 service neutron] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:26:54.478116 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4a7488-bfba-48b6-aaca-67f3b711bf63 service neutron] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:26:54.478452 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4a7488-bfba-48b6-aaca-67f3b711bf63 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:26:54.480220 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-9b4a7488-bfba-48b6-aaca-67f3b711bf63 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:26:54.490016 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4a7488-bfba-48b6-aaca-67f3b711bf63 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:26:54.491452 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4a7488-bfba-48b6-aaca-67f3b711bf63 service neutron] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:26:54.499475 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1383/2743] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:26:54 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:26:55.521868 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=15 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:26:58.013759 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e1e903-bdbf-4519-9573-3f17fa517d55 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:26:58.015339 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-33e1e903-bdbf-4519-9573-3f17fa517d55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:26:58.093275 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e1e903-bdbf-4519-9573-3f17fa517d55 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:26:58.099156 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da631572-1b37-4213-b474-0eaa26395ef4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15e7f45ee0744c1d9602187eac8e8b80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:26:58.099825 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da631572-1b37-4213-b474-0eaa26395ef4 service nova] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:26:58.100115 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da631572-1b37-4213-b474-0eaa26395ef4 service nova] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:26:58.100388 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da631572-1b37-4213-b474-0eaa26395ef4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:26:58.102276 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-da631572-1b37-4213-b474-0eaa26395ef4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:26:58.115848 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da631572-1b37-4213-b474-0eaa26395ef4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:26:58.117405 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da631572-1b37-4213-b474-0eaa26395ef4 service nova] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:26:58.129792 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1361/2744] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:26:58 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:27:04.642071 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=17 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:27:08.124138 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=34 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:27:54.689316 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4a7488-bfba-48b6-aaca-67f3b711bf63 service neutron] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:27:54.700152 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4a7488-bfba-48b6-aaca-67f3b711bf63 service neutron] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:27:54.704214 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6011ae5-fa28-458a-b0cf-7c998d300fa7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c4d7cde8f4ad9aa479bb09e325db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:27:54.704851 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6011ae5-fa28-458a-b0cf-7c998d300fa7 service neutron] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:27:54.705133 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6011ae5-fa28-458a-b0cf-7c998d300fa7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:27:54.705436 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6011ae5-fa28-458a-b0cf-7c998d300fa7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:27:54.707201 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-f6011ae5-fa28-458a-b0cf-7c998d300fa7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:27:54.771068 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6011ae5-fa28-458a-b0cf-7c998d300fa7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:27:54.773502 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6011ae5-fa28-458a-b0cf-7c998d300fa7 service neutron] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:27:54.782297 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1384/2745] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:27:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:27:55.752349 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2f57d4-b665-4689-af66-cdf1815cdcf7 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:27:55.753280 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2f57d4-b665-4689-af66-cdf1815cdcf7 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:27:55.753736 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2f57d4-b665-4689-af66-cdf1815cdcf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:27:55.773475 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-eb2f57d4-b665-4689-af66-cdf1815cdcf7 None None] MFA Rules not processed for user `e5dc9999f6664c23855b66670f7d6a98`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:27:55.774919 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-eb2f57d4-b665-4689-af66-cdf1815cdcf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:27:55.787229 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1362/2746] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:27:55 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:27:56.625601 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6011ae5-fa28-458a-b0cf-7c998d300fa7 service neutron] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:27:56.627700 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-f6011ae5-fa28-458a-b0cf-7c998d300fa7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:27:56.727731 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6011ae5-fa28-458a-b0cf-7c998d300fa7 service neutron] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:27:56.733302 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24380af3-e5a0-4f85-b232-163dfd154734 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5323ffd91f864680b0c0084eb3f44d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:27:56.734050 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24380af3-e5a0-4f85-b232-163dfd154734 service placement] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:27:56.734409 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24380af3-e5a0-4f85-b232-163dfd154734 service placement] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:27:56.734758 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24380af3-e5a0-4f85-b232-163dfd154734 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:27:56.736686 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-24380af3-e5a0-4f85-b232-163dfd154734 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:27:56.752616 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24380af3-e5a0-4f85-b232-163dfd154734 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:27:56.754297 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24380af3-e5a0-4f85-b232-163dfd154734 service placement] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:27:56.766410 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1385/2747] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:27:56 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:28:02.124707 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2f57d4-b665-4689-af66-cdf1815cdcf7 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:28:02.126283 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-eb2f57d4-b665-4689-af66-cdf1815cdcf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:28:02.148076 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2f57d4-b665-4689-af66-cdf1815cdcf7 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:28:02.156527 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890b966a-d792-48ab-8171-db0277688e74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:28:02.157607 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890b966a-d792-48ab-8171-db0277688e74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:28:02.158147 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890b966a-d792-48ab-8171-db0277688e74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:28:02.158454 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890b966a-d792-48ab-8171-db0277688e74 admin admin] PATH_INFO: `/v3/users/6d24c8fbdc7246e2af87004f2985a9f5` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:28:02.159721 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890b966a-d792-48ab-8171-db0277688e74 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d24c8fbdc7246e2af87004f2985a9f5)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:28:02.164067 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890b966a-d792-48ab-8171-db0277688e74 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:28:02.377560 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-890b966a-d792-48ab-8171-db0277688e74 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d24c8fbdc7246e2af87004f2985a9f5'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:28:02.378273 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-890b966a-d792-48ab-8171-db0277688e74 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d24c8fbdc7246e2af87004f2985a9f5'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:28:02.451974 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-890b966a-d792-48ab-8171-db0277688e74 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d24c8fbdc7246e2af87004f2985a9f5'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:28:02.466648 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-890b966a-d792-48ab-8171-db0277688e74 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d24c8fbdc7246e2af87004f2985a9f5'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:28:02.473705 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-890b966a-d792-48ab-8171-db0277688e74 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d24c8fbdc7246e2af87004f2985a9f5'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:28:02.481091 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1363/2748] 10.222.0.23 () {58 vars in 1223 bytes} [Sat Jun 18 05:28:02 2022] DELETE /identity/v3/users/6d24c8fbdc7246e2af87004f2985a9f5 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:28:02.753354 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24380af3-e5a0-4f85-b232-163dfd154734 service placement] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:28:02.773647 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24380af3-e5a0-4f85-b232-163dfd154734 service placement] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:28:02.839024 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a086bf4e-f18c-4470-9be8-00e689c0026c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:28:02.840033 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a086bf4e-f18c-4470-9be8-00e689c0026c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:28:02.840389 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a086bf4e-f18c-4470-9be8-00e689c0026c admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:28:02.840735 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a086bf4e-f18c-4470-9be8-00e689c0026c admin admin] PATH_INFO: `/v3/projects/e5e4aece32ff462b9b9a42200d584c1b` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:28:02.842078 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a086bf4e-f18c-4470-9be8-00e689c0026c admin admin] RBAC: Authorizing `identity:delete_project(project_id=e5e4aece32ff462b9b9a42200d584c1b)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:28:02.844103 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a086bf4e-f18c-4470-9be8-00e689c0026c admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:28:02.950385 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:28:02.967508 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-a086bf4e-f18c-4470-9be8-00e689c0026c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e5e4aece32ff462b9b9a42200d584c1b', 'request_id': 'req-a086bf4e-f18c-4470-9be8-00e689c0026c'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:28:02.972901 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-a086bf4e-f18c-4470-9be8-00e689c0026c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e5e4aece32ff462b9b9a42200d584c1b', 'request_id': 'req-a086bf4e-f18c-4470-9be8-00e689c0026c'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:28:02.979521 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-a086bf4e-f18c-4470-9be8-00e689c0026c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e5e4aece32ff462b9b9a42200d584c1b', 'request_id': 'req-a086bf4e-f18c-4470-9be8-00e689c0026c'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:28:03.080999 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-a086bf4e-f18c-4470-9be8-00e689c0026c admin admin] The token cache is being invalidate because project e5e4aece32ff462b9b9a42200d584c1b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80469) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 18 05:28:03.081331 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-a086bf4e-f18c-4470-9be8-00e689c0026c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:28:03.081933 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-a086bf4e-f18c-4470-9be8-00e689c0026c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:28:03.083472 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1386/2749] 10.222.0.23 () {58 vars in 1232 bytes} [Sat Jun 18 05:28:02 2022] DELETE /identity/v3/projects/e5e4aece32ff462b9b9a42200d584c1b => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:28:03.087629 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890b966a-d792-48ab-8171-db0277688e74 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:28:03.089048 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-890b966a-d792-48ab-8171-db0277688e74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:28:03.102498 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890b966a-d792-48ab-8171-db0277688e74 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:28:03.162826 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c71561-c971-42ae-9b2c-25f7dd249184 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:28:03.163704 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c71561-c971-42ae-9b2c-25f7dd249184 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:28:03.164283 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c71561-c971-42ae-9b2c-25f7dd249184 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:28:03.164283 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c71561-c971-42ae-9b2c-25f7dd249184 admin admin] PATH_INFO: `/v3/users/cd400db0aacb44389379f5c424b81237` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:28:03.165426 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c71561-c971-42ae-9b2c-25f7dd249184 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd400db0aacb44389379f5c424b81237)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:28:03.167419 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c71561-c971-42ae-9b2c-25f7dd249184 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:28:03.377302 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-53c71561-c971-42ae-9b2c-25f7dd249184 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd400db0aacb44389379f5c424b81237'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:28:03.378325 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-53c71561-c971-42ae-9b2c-25f7dd249184 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd400db0aacb44389379f5c424b81237'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:28:03.453508 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-53c71561-c971-42ae-9b2c-25f7dd249184 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd400db0aacb44389379f5c424b81237'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:28:03.471291 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:28:03.473882 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-53c71561-c971-42ae-9b2c-25f7dd249184 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd400db0aacb44389379f5c424b81237'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:28:03.483804 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-53c71561-c971-42ae-9b2c-25f7dd249184 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd400db0aacb44389379f5c424b81237'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:28:03.508982 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1364/2750] 10.222.0.23 () {58 vars in 1223 bytes} [Sat Jun 18 05:28:03 2022] DELETE /identity/v3/users/cd400db0aacb44389379f5c424b81237 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:28:03.765256 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a086bf4e-f18c-4470-9be8-00e689c0026c admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:28:03.766810 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-a086bf4e-f18c-4470-9be8-00e689c0026c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:28:03.785680 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a086bf4e-f18c-4470-9be8-00e689c0026c admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:28:03.854940 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445dc905-7b28-4280-bd70-bfd5d98e29dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:28:03.855945 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445dc905-7b28-4280-bd70-bfd5d98e29dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:28:03.856397 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445dc905-7b28-4280-bd70-bfd5d98e29dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:28:03.856569 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445dc905-7b28-4280-bd70-bfd5d98e29dd admin admin] PATH_INFO: `/v3/projects/9098492a9e6644edb4d02b2b4b6399ad` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:28:03.857841 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445dc905-7b28-4280-bd70-bfd5d98e29dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=9098492a9e6644edb4d02b2b4b6399ad)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:28:03.861231 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445dc905-7b28-4280-bd70-bfd5d98e29dd admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:28:04.020164 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-445dc905-7b28-4280-bd70-bfd5d98e29dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9098492a9e6644edb4d02b2b4b6399ad', 'request_id': 'req-445dc905-7b28-4280-bd70-bfd5d98e29dd'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:28:04.026891 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-445dc905-7b28-4280-bd70-bfd5d98e29dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9098492a9e6644edb4d02b2b4b6399ad', 'request_id': 'req-445dc905-7b28-4280-bd70-bfd5d98e29dd'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:28:04.033404 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-445dc905-7b28-4280-bd70-bfd5d98e29dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9098492a9e6644edb4d02b2b4b6399ad', 'request_id': 'req-445dc905-7b28-4280-bd70-bfd5d98e29dd'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:28:04.116067 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-445dc905-7b28-4280-bd70-bfd5d98e29dd admin admin] The token cache is being invalidate because project 9098492a9e6644edb4d02b2b4b6399ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80469) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 18 05:28:04.116390 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-445dc905-7b28-4280-bd70-bfd5d98e29dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:28:04.117258 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-445dc905-7b28-4280-bd70-bfd5d98e29dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:28:04.118859 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1387/2751] 10.222.0.23 () {58 vars in 1232 bytes} [Sat Jun 18 05:28:03 2022] DELETE /identity/v3/projects/9098492a9e6644edb4d02b2b4b6399ad => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 18 05:28:08.486103 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c71561-c971-42ae-9b2c-25f7dd249184 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:28:08.486103 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-53c71561-c971-42ae-9b2c-25f7dd249184 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:28:08.556706 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c71561-c971-42ae-9b2c-25f7dd249184 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:28:08.622662 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d2abed-5f5a-445e-a2d3-10449ad9183e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15e7f45ee0744c1d9602187eac8e8b80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:28:08.623353 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d2abed-5f5a-445e-a2d3-10449ad9183e service nova] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:28:08.623636 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d2abed-5f5a-445e-a2d3-10449ad9183e service nova] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:28:08.623911 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d2abed-5f5a-445e-a2d3-10449ad9183e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:28:08.625791 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-31d2abed-5f5a-445e-a2d3-10449ad9183e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:28:08.638157 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d2abed-5f5a-445e-a2d3-10449ad9183e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:28:08.639763 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d2abed-5f5a-445e-a2d3-10449ad9183e service nova] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:28:08.651654 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1365/2752] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:28:08 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:28:14.113154 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:DELETE=13,keystone:SELECT=65,keystone:INSERT=2 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:28:18.645912 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:DELETE=1,keystone:SELECT=68 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:28:55.935640 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445dc905-7b28-4280-bd70-bfd5d98e29dd admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:28:55.937409 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-445dc905-7b28-4280-bd70-bfd5d98e29dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:28:55.995935 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445dc905-7b28-4280-bd70-bfd5d98e29dd admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:28:55.999911 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1530ea-2715-40d7-b4d6-ff2c7d16eb28 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9de0428abc8400aa338aba4f7c34c94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:28:56.000347 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1530ea-2715-40d7-b4d6-ff2c7d16eb28 service cinder] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:28:56.000634 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1530ea-2715-40d7-b4d6-ff2c7d16eb28 service cinder] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:28:56.000911 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1530ea-2715-40d7-b4d6-ff2c7d16eb28 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:28:56.002732 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-cb1530ea-2715-40d7-b4d6-ff2c7d16eb28 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:28:56.064662 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1530ea-2715-40d7-b4d6-ff2c7d16eb28 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:28:56.066158 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1530ea-2715-40d7-b4d6-ff2c7d16eb28 service cinder] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:28:56.075189 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1388/2753] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:28:55 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:29:01.707130 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d2abed-5f5a-445e-a2d3-10449ad9183e service nova] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:29:01.708723 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-31d2abed-5f5a-445e-a2d3-10449ad9183e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:29:01.779385 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d2abed-5f5a-445e-a2d3-10449ad9183e service nova] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:29:01.786901 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6ca4b4-fd70-4bbe-afea-1ec15de80844 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15e7f45ee0744c1d9602187eac8e8b80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:29:01.787327 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6ca4b4-fd70-4bbe-afea-1ec15de80844 service nova] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:29:01.791190 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6ca4b4-fd70-4bbe-afea-1ec15de80844 service nova] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:29:01.793097 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6ca4b4-fd70-4bbe-afea-1ec15de80844 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:29:01.793727 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-9e6ca4b4-fd70-4bbe-afea-1ec15de80844 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:29:01.809512 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6ca4b4-fd70-4bbe-afea-1ec15de80844 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:29:01.811118 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6ca4b4-fd70-4bbe-afea-1ec15de80844 service nova] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:29:01.824873 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1366/2754] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:29:01 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:29:06.070991 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=48 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:29:08.660161 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1530ea-2715-40d7-b4d6-ff2c7d16eb28 service cinder] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:29:08.661658 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-cb1530ea-2715-40d7-b4d6-ff2c7d16eb28 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:29:08.738084 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1530ea-2715-40d7-b4d6-ff2c7d16eb28 service cinder] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:29:08.741830 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63e0565-bfdc-4279-90a2-27ef43689044 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c4d7cde8f4ad9aa479bb09e325db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:29:08.742471 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63e0565-bfdc-4279-90a2-27ef43689044 service neutron] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:29:08.742755 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63e0565-bfdc-4279-90a2-27ef43689044 service neutron] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:29:08.743033 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63e0565-bfdc-4279-90a2-27ef43689044 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:29:08.744857 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-f63e0565-bfdc-4279-90a2-27ef43689044 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:29:08.828460 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63e0565-bfdc-4279-90a2-27ef43689044 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:29:08.829981 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63e0565-bfdc-4279-90a2-27ef43689044 service neutron] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:29:08.849265 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1389/2755] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:29:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:29:11.820037 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=28 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:29:12.459906 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6ca4b4-fd70-4bbe-afea-1ec15de80844 service nova] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:29:12.461306 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-9e6ca4b4-fd70-4bbe-afea-1ec15de80844 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:29:12.531407 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6ca4b4-fd70-4bbe-afea-1ec15de80844 service nova] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:29:12.536755 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3733f71-4105-4e5d-a02f-76b0015636e3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5323ffd91f864680b0c0084eb3f44d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:29:12.537457 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3733f71-4105-4e5d-a02f-76b0015636e3 service placement] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:29:12.537748 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3733f71-4105-4e5d-a02f-76b0015636e3 service placement] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:29:12.538028 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3733f71-4105-4e5d-a02f-76b0015636e3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:29:12.539820 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-a3733f71-4105-4e5d-a02f-76b0015636e3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:29:12.551576 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3733f71-4105-4e5d-a02f-76b0015636e3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:29:12.553150 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3733f71-4105-4e5d-a02f-76b0015636e3 service placement] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:29:12.563956 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1367/2756] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:29:12 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:29:12.598291 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63e0565-bfdc-4279-90a2-27ef43689044 service neutron] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:29:12.599909 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-f63e0565-bfdc-4279-90a2-27ef43689044 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:29:12.617534 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63e0565-bfdc-4279-90a2-27ef43689044 service neutron] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:29:12.619833 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbc6b33-d5b8-49aa-8891-d39677ef778d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:29:12.620701 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbc6b33-d5b8-49aa-8891-d39677ef778d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:29:12.620996 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbc6b33-d5b8-49aa-8891-d39677ef778d admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:29:12.621250 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbc6b33-d5b8-49aa-8891-d39677ef778d admin admin] PATH_INFO: `/v3/users/e5dc9999f6664c23855b66670f7d6a98` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:29:12.622524 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbc6b33-d5b8-49aa-8891-d39677ef778d admin admin] RBAC: Authorizing `identity:delete_user(user_id=e5dc9999f6664c23855b66670f7d6a98)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:29:12.624525 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbc6b33-d5b8-49aa-8891-d39677ef778d admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:29:12.797369 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-1bbc6b33-d5b8-49aa-8891-d39677ef778d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e5dc9999f6664c23855b66670f7d6a98'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:29:12.804474 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:29:12.809801 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-1bbc6b33-d5b8-49aa-8891-d39677ef778d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e5dc9999f6664c23855b66670f7d6a98'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:29:12.810465 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-1bbc6b33-d5b8-49aa-8891-d39677ef778d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e5dc9999f6664c23855b66670f7d6a98'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:29:12.898078 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-1bbc6b33-d5b8-49aa-8891-d39677ef778d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e5dc9999f6664c23855b66670f7d6a98'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:29:12.905099 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-1bbc6b33-d5b8-49aa-8891-d39677ef778d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e5dc9999f6664c23855b66670f7d6a98'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:29:12.911310 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1390/2757] 10.222.0.23 () {58 vars in 1223 bytes} [Sat Jun 18 05:29:12 2022] DELETE /identity/v3/users/e5dc9999f6664c23855b66670f7d6a98 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:29:13.166763 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3733f71-4105-4e5d-a02f-76b0015636e3 service placement] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:29:13.179762 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3733f71-4105-4e5d-a02f-76b0015636e3 service placement] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:29:13.238414 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3862ce35-969a-42e2-8696-805e67976e4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:29:13.239260 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3862ce35-969a-42e2-8696-805e67976e4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:29:13.239539 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3862ce35-969a-42e2-8696-805e67976e4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:29:13.239810 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3862ce35-969a-42e2-8696-805e67976e4d admin admin] PATH_INFO: `/v3/projects/81c38b2c1ae4484bbf2d4939470f45e4` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:29:13.240979 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3862ce35-969a-42e2-8696-805e67976e4d admin admin] RBAC: Authorizing `identity:delete_project(project_id=81c38b2c1ae4484bbf2d4939470f45e4)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:29:13.242923 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3862ce35-969a-42e2-8696-805e67976e4d admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:29:13.376300 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-3862ce35-969a-42e2-8696-805e67976e4d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '81c38b2c1ae4484bbf2d4939470f45e4', 'request_id': 'req-3862ce35-969a-42e2-8696-805e67976e4d'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:29:13.382048 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-3862ce35-969a-42e2-8696-805e67976e4d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '81c38b2c1ae4484bbf2d4939470f45e4', 'request_id': 'req-3862ce35-969a-42e2-8696-805e67976e4d'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:29:13.450211 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-3862ce35-969a-42e2-8696-805e67976e4d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '81c38b2c1ae4484bbf2d4939470f45e4', 'request_id': 'req-3862ce35-969a-42e2-8696-805e67976e4d'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:29:13.462323 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-3862ce35-969a-42e2-8696-805e67976e4d admin admin] The token cache is being invalidate because project 81c38b2c1ae4484bbf2d4939470f45e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80468) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 18 05:29:13.462612 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-3862ce35-969a-42e2-8696-805e67976e4d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:29:13.463244 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-3862ce35-969a-42e2-8696-805e67976e4d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:29:13.465025 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1368/2758] 10.222.0.23 () {58 vars in 1232 bytes} [Sat Jun 18 05:29:13 2022] DELETE /identity/v3/projects/81c38b2c1ae4484bbf2d4939470f45e4 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:29:13.469819 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18538303-6836-4a82-8b17-e00a6a102a97 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:29:13.472515 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18538303-6836-4a82-8b17-e00a6a102a97 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:29:13.472515 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18538303-6836-4a82-8b17-e00a6a102a97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:29:13.530958 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-18538303-6836-4a82-8b17-e00a6a102a97 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:29:13.532734 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-18538303-6836-4a82-8b17-e00a6a102a97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:29:13.594306 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1391/2759] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:29:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:29:13.602131 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3e2769-775f-49a7-9f01-dcd12f9aa252 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:29:13.602291 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3e2769-775f-49a7-9f01-dcd12f9aa252 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:29:13.602601 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3e2769-775f-49a7-9f01-dcd12f9aa252 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:29:13.623101 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-8b3e2769-775f-49a7-9f01-dcd12f9aa252 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:29:13.624307 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-8b3e2769-775f-49a7-9f01-dcd12f9aa252 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:29:13.632995 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1369/2760] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:29:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:29:13.637138 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18538303-6836-4a82-8b17-e00a6a102a97 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:29:13.647365 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18538303-6836-4a82-8b17-e00a6a102a97 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:29:13.652379 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb9ef4a-366c-4958-a0ce-5c601c403ef3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:29:13.653037 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb9ef4a-366c-4958-a0ce-5c601c403ef3 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:29:13.653323 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb9ef4a-366c-4958-a0ce-5c601c403ef3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:29:13.653635 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb9ef4a-366c-4958-a0ce-5c601c403ef3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:29:13.654422 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb9ef4a-366c-4958-a0ce-5c601c403ef3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 05:29:13.654735 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb9ef4a-366c-4958-a0ce-5c601c403ef3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:29:13.656341 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb9ef4a-366c-4958-a0ce-5c601c403ef3 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:29:13.668995 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1392/2761] 10.222.0.23 () {58 vars in 1152 bytes} [Sat Jun 18 05:29:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 05:29:13.673614 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3e2769-775f-49a7-9f01-dcd12f9aa252 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:29:13.682262 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3e2769-775f-49a7-9f01-dcd12f9aa252 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:29:13.686879 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca81239-5ecc-41c4-8f14-c4bd1a4d7440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:29:13.687517 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca81239-5ecc-41c4-8f14-c4bd1a4d7440 admin admin] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:29:13.687795 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca81239-5ecc-41c4-8f14-c4bd1a4d7440 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:29:13.688340 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca81239-5ecc-41c4-8f14-c4bd1a4d7440 admin admin] PATH_INFO: `/v3/projects` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:29:13.688750 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca81239-5ecc-41c4-8f14-c4bd1a4d7440 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:29:13.690659 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca81239-5ecc-41c4-8f14-c4bd1a4d7440 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:29:13.700427 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:29:13.776994 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1370/2762] 10.222.0.23 () {60 vars in 1152 bytes} [Sat Jun 18 05:29:13 2022] POST /identity/v3/projects => generated 373 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:29:13.781060 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb9ef4a-366c-4958-a0ce-5c601c403ef3 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:29:13.791775 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb9ef4a-366c-4958-a0ce-5c601c403ef3 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:29:13.848528 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ecf217-86e5-4053-9248-bed004325237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:29:13.849184 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ecf217-86e5-4053-9248-bed004325237 admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:29:13.849498 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ecf217-86e5-4053-9248-bed004325237 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:29:13.849771 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ecf217-86e5-4053-9248-bed004325237 admin admin] PATH_INFO: `/v3/users` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:29:13.850495 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ecf217-86e5-4053-9248-bed004325237 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:29:13.852397 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ecf217-86e5-4053-9248-bed004325237 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:29:13.955541 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1393/2763] 10.222.0.23 () {60 vars in 1143 bytes} [Sat Jun 18 05:29:13 2022] POST /identity/v3/users => generated 287 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:29:13.959633 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca81239-5ecc-41c4-8f14-c4bd1a4d7440 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:29:13.970480 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca81239-5ecc-41c4-8f14-c4bd1a4d7440 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:29:13.975178 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e202970-c91e-43e3-91f6-a3f801a2b026 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:29:13.975825 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e202970-c91e-43e3-91f6-a3f801a2b026 admin admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:29:13.976108 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e202970-c91e-43e3-91f6-a3f801a2b026 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:29:13.976380 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e202970-c91e-43e3-91f6-a3f801a2b026 admin admin] PATH_INFO: `/v3/roles` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:29:13.977099 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e202970-c91e-43e3-91f6-a3f801a2b026 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:29:13.978719 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e202970-c91e-43e3-91f6-a3f801a2b026 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:29:13.989605 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1371/2764] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:29:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:29:13.993570 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ecf217-86e5-4053-9248-bed004325237 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:29:14.004451 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ecf217-86e5-4053-9248-bed004325237 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:29:14.009095 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7363429d-023d-4154-80bc-12b65806fc23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:29:14.009814 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7363429d-023d-4154-80bc-12b65806fc23 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:29:14.010101 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7363429d-023d-4154-80bc-12b65806fc23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:29:14.010377 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7363429d-023d-4154-80bc-12b65806fc23 admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:29:14.011099 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7363429d-023d-4154-80bc-12b65806fc23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:29:14.012684 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7363429d-023d-4154-80bc-12b65806fc23 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:29:14.022713 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1394/2765] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:29:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:29:14.026497 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e202970-c91e-43e3-91f6-a3f801a2b026 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:29:14.037016 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e202970-c91e-43e3-91f6-a3f801a2b026 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:29:14.041647 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4627ef6d-d005-4af1-9eac-3cd46f404973 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:29:14.042499 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4627ef6d-d005-4af1-9eac-3cd46f404973 admin admin] REQUEST_METHOD: `PUT` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:29:14.042782 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4627ef6d-d005-4af1-9eac-3cd46f404973 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:29:14.043059 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4627ef6d-d005-4af1-9eac-3cd46f404973 admin admin] PATH_INFO: `/v3/projects/98b2c0715f93458785782dfe3574d15c/users/a1ab6b547b02449c9d9e2026df4aab0d/roles/cbf5c4c26898413d93e73b46b41046a3` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:29:14.058053 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4627ef6d-d005-4af1-9eac-3cd46f404973 admin admin] RBAC: Authorizing `identity:create_grant(project_id=98b2c0715f93458785782dfe3574d15c, user_id=a1ab6b547b02449c9d9e2026df4aab0d, role_id=cbf5c4c26898413d93e73b46b41046a3)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:29:14.060749 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4627ef6d-d005-4af1-9eac-3cd46f404973 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:29:14.163015 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1372/2766] 10.222.0.23 () {60 vars in 1482 bytes} [Sat Jun 18 05:29:14 2022] PUT /identity/v3/projects/98b2c0715f93458785782dfe3574d15c/users/a1ab6b547b02449c9d9e2026df4aab0d/roles/cbf5c4c26898413d93e73b46b41046a3 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:29:14.174252 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0f2f49-a3a3-446e-b215-07709dca16dd None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:29:14.174581 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0f2f49-a3a3-446e-b215-07709dca16dd None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:29:14.174866 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0f2f49-a3a3-446e-b215-07709dca16dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:29:14.193933 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:29:14.202117 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-fa0f2f49-a3a3-446e-b215-07709dca16dd None None] MFA Rules not processed for user `a1ab6b547b02449c9d9e2026df4aab0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:29:14.203323 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-fa0f2f49-a3a3-446e-b215-07709dca16dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:29:14.266776 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1395/2767] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:29:14 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:29:14.277704 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4627ef6d-d005-4af1-9eac-3cd46f404973 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:29:14.279469 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-4627ef6d-d005-4af1-9eac-3cd46f404973 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:29:14.352743 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4627ef6d-d005-4af1-9eac-3cd46f404973 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:29:14.358523 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3958db5a-48dc-40d8-a71b-6ca22802a1a7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15e7f45ee0744c1d9602187eac8e8b80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:29:14.359158 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3958db5a-48dc-40d8-a71b-6ca22802a1a7 service nova] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:29:14.359446 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3958db5a-48dc-40d8-a71b-6ca22802a1a7 service nova] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:29:14.360262 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3958db5a-48dc-40d8-a71b-6ca22802a1a7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:29:14.368102 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3958db5a-48dc-40d8-a71b-6ca22802a1a7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:29:14.369713 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3958db5a-48dc-40d8-a71b-6ca22802a1a7 service nova] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:29:14.378668 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1373/2768] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:29:14 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:29:17.497130 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0f2f49-a3a3-446e-b215-07709dca16dd None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:29:17.558046 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0f2f49-a3a3-446e-b215-07709dca16dd None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:29:17.562615 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8450127-1a38-4990-a432-41322a540215 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:29:17.564196 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8450127-1a38-4990-a432-41322a540215 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:29:17.564479 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8450127-1a38-4990-a432-41322a540215 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:29:17.564753 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8450127-1a38-4990-a432-41322a540215 admin admin] PATH_INFO: `/v3/users/a1ab6b547b02449c9d9e2026df4aab0d` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:29:17.565973 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8450127-1a38-4990-a432-41322a540215 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1ab6b547b02449c9d9e2026df4aab0d)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:29:17.567902 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8450127-1a38-4990-a432-41322a540215 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:29:17.762808 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-e8450127-1a38-4990-a432-41322a540215 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1ab6b547b02449c9d9e2026df4aab0d'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:29:17.774964 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-e8450127-1a38-4990-a432-41322a540215 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1ab6b547b02449c9d9e2026df4aab0d'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:29:17.775626 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-e8450127-1a38-4990-a432-41322a540215 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1ab6b547b02449c9d9e2026df4aab0d'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:29:17.835086 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-e8450127-1a38-4990-a432-41322a540215 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1ab6b547b02449c9d9e2026df4aab0d'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:29:17.842068 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-e8450127-1a38-4990-a432-41322a540215 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1ab6b547b02449c9d9e2026df4aab0d'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:29:17.848963 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1396/2769] 10.222.0.23 () {58 vars in 1223 bytes} [Sat Jun 18 05:29:17 2022] DELETE /identity/v3/users/a1ab6b547b02449c9d9e2026df4aab0d => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:29:17.853369 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ef8d29-6e77-4e5b-9566-b343ed9f44a0 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:29:17.853886 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ef8d29-6e77-4e5b-9566-b343ed9f44a0 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:29:17.854010 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ef8d29-6e77-4e5b-9566-b343ed9f44a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:29:17.871871 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-f7ef8d29-6e77-4e5b-9566-b343ed9f44a0 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:29:17.873102 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-f7ef8d29-6e77-4e5b-9566-b343ed9f44a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:29:17.933240 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1374/2770] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:29:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:29:17.943422 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8450127-1a38-4990-a432-41322a540215 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:29:17.944846 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-e8450127-1a38-4990-a432-41322a540215 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:29:17.966688 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:29:18.024198 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8450127-1a38-4990-a432-41322a540215 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:29:18.027883 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14964ad9-d6ad-4a5c-8a33-17ea66a585b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c4d7cde8f4ad9aa479bb09e325db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:29:18.028524 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14964ad9-d6ad-4a5c-8a33-17ea66a585b4 service neutron] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:29:18.028815 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14964ad9-d6ad-4a5c-8a33-17ea66a585b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:29:18.029118 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14964ad9-d6ad-4a5c-8a33-17ea66a585b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:29:18.039618 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14964ad9-d6ad-4a5c-8a33-17ea66a585b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:29:18.041064 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14964ad9-d6ad-4a5c-8a33-17ea66a585b4 service neutron] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:29:18.051444 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1397/2771] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:29:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:29:18.448729 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ef8d29-6e77-4e5b-9566-b343ed9f44a0 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:29:18.450309 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-f7ef8d29-6e77-4e5b-9566-b343ed9f44a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:29:18.462990 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ef8d29-6e77-4e5b-9566-b343ed9f44a0 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:29:18.467617 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafe5421-9d67-44eb-b499-c110b524e65c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:29:18.468480 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafe5421-9d67-44eb-b499-c110b524e65c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:29:18.468768 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafe5421-9d67-44eb-b499-c110b524e65c admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:29:18.469046 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafe5421-9d67-44eb-b499-c110b524e65c admin admin] PATH_INFO: `/v3/projects/98b2c0715f93458785782dfe3574d15c` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:29:18.470974 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafe5421-9d67-44eb-b499-c110b524e65c admin admin] RBAC: Authorizing `identity:delete_project(project_id=98b2c0715f93458785782dfe3574d15c)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:29:18.472901 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafe5421-9d67-44eb-b499-c110b524e65c admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:29:18.565310 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:29:18.573782 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-aafe5421-9d67-44eb-b499-c110b524e65c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '98b2c0715f93458785782dfe3574d15c', 'request_id': 'req-aafe5421-9d67-44eb-b499-c110b524e65c'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:29:18.579851 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-aafe5421-9d67-44eb-b499-c110b524e65c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '98b2c0715f93458785782dfe3574d15c', 'request_id': 'req-aafe5421-9d67-44eb-b499-c110b524e65c'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:29:18.627510 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-aafe5421-9d67-44eb-b499-c110b524e65c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '98b2c0715f93458785782dfe3574d15c', 'request_id': 'req-aafe5421-9d67-44eb-b499-c110b524e65c'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:29:18.639117 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-aafe5421-9d67-44eb-b499-c110b524e65c admin admin] The token cache is being invalidate because project 98b2c0715f93458785782dfe3574d15c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80468) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 18 05:29:18.639602 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-aafe5421-9d67-44eb-b499-c110b524e65c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:29:18.640615 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-aafe5421-9d67-44eb-b499-c110b524e65c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:29:18.642917 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1375/2772] 10.222.0.23 () {58 vars in 1232 bytes} [Sat Jun 18 05:29:18 2022] DELETE /identity/v3/projects/98b2c0715f93458785782dfe3574d15c => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:29:18.647956 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eae75f-4011-4493-abf4-5d5b1cf59f1f None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:29:18.648450 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eae75f-4011-4493-abf4-5d5b1cf59f1f None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:29:18.648634 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eae75f-4011-4493-abf4-5d5b1cf59f1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:29:18.668113 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-17eae75f-4011-4493-abf4-5d5b1cf59f1f None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:29:18.669313 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-17eae75f-4011-4493-abf4-5d5b1cf59f1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:29:18.727139 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1398/2773] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:29:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:29:18.735075 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0f57f6-26cd-44d2-9238-12f1e5815ba9 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:29:18.735367 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0f57f6-26cd-44d2-9238-12f1e5815ba9 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:29:18.735652 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0f57f6-26cd-44d2-9238-12f1e5815ba9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:29:18.754519 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-ee0f57f6-26cd-44d2-9238-12f1e5815ba9 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:29:18.755731 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-ee0f57f6-26cd-44d2-9238-12f1e5815ba9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:29:18.763597 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1376/2774] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:29:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:29:18.768415 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17eae75f-4011-4493-abf4-5d5b1cf59f1f None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:29:18.778103 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17eae75f-4011-4493-abf4-5d5b1cf59f1f None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:29:18.782715 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286424ad-8928-47cc-98a2-e7559992dbfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:29:18.783350 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286424ad-8928-47cc-98a2-e7559992dbfe admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:29:18.783629 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286424ad-8928-47cc-98a2-e7559992dbfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:29:18.784152 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286424ad-8928-47cc-98a2-e7559992dbfe admin admin] PATH_INFO: `/v3/domains` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:29:18.784869 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286424ad-8928-47cc-98a2-e7559992dbfe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 05:29:18.785179 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286424ad-8928-47cc-98a2-e7559992dbfe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:29:18.786809 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286424ad-8928-47cc-98a2-e7559992dbfe admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:29:18.798223 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1399/2775] 10.222.0.23 () {58 vars in 1152 bytes} [Sat Jun 18 05:29:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 05:29:18.802332 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0f57f6-26cd-44d2-9238-12f1e5815ba9 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:29:18.811503 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0f57f6-26cd-44d2-9238-12f1e5815ba9 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:29:18.816695 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf70532-c2c8-499d-8f7e-e76fee956dd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:29:18.817282 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf70532-c2c8-499d-8f7e-e76fee956dd7 admin admin] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:29:18.817592 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf70532-c2c8-499d-8f7e-e76fee956dd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:29:18.817875 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf70532-c2c8-499d-8f7e-e76fee956dd7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:29:18.818563 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf70532-c2c8-499d-8f7e-e76fee956dd7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:29:18.820452 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf70532-c2c8-499d-8f7e-e76fee956dd7 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:29:18.886103 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1377/2776] 10.222.0.23 () {60 vars in 1152 bytes} [Sat Jun 18 05:29:18 2022] POST /identity/v3/projects => generated 383 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:29:18.890335 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286424ad-8928-47cc-98a2-e7559992dbfe admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:29:18.900721 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286424ad-8928-47cc-98a2-e7559992dbfe admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:29:18.958032 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaaeb07-db14-4d83-9e1d-2deb89e95816 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:29:18.958709 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaaeb07-db14-4d83-9e1d-2deb89e95816 admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:29:18.958995 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaaeb07-db14-4d83-9e1d-2deb89e95816 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:29:18.959269 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaaeb07-db14-4d83-9e1d-2deb89e95816 admin admin] PATH_INFO: `/v3/users` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:29:18.959996 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddaaeb07-db14-4d83-9e1d-2deb89e95816 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:29:18.961945 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddaaeb07-db14-4d83-9e1d-2deb89e95816 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:29:19.092765 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1400/2777] 10.222.0.23 () {60 vars in 1143 bytes} [Sat Jun 18 05:29:18 2022] POST /identity/v3/users => generated 292 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:29:19.096848 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf70532-c2c8-499d-8f7e-e76fee956dd7 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:29:19.107237 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf70532-c2c8-499d-8f7e-e76fee956dd7 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:29:19.112682 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e67cab-959f-4a6f-8b1c-65e2e804f4e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:29:19.113339 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e67cab-959f-4a6f-8b1c-65e2e804f4e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:29:19.113689 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e67cab-959f-4a6f-8b1c-65e2e804f4e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:29:19.113969 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e67cab-959f-4a6f-8b1c-65e2e804f4e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:29:19.114698 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e67cab-959f-4a6f-8b1c-65e2e804f4e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:29:19.116285 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e67cab-959f-4a6f-8b1c-65e2e804f4e8 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:29:19.127199 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1378/2778] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:29:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:29:19.131053 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaaeb07-db14-4d83-9e1d-2deb89e95816 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:29:19.141212 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaaeb07-db14-4d83-9e1d-2deb89e95816 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:29:19.145787 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5ec2bf-0e25-4ae9-925c-9638d3a05611 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:29:19.146436 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5ec2bf-0e25-4ae9-925c-9638d3a05611 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:29:19.146719 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5ec2bf-0e25-4ae9-925c-9638d3a05611 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:29:19.146994 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5ec2bf-0e25-4ae9-925c-9638d3a05611 admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:29:19.147716 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5ec2bf-0e25-4ae9-925c-9638d3a05611 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:29:19.149311 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5ec2bf-0e25-4ae9-925c-9638d3a05611 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:29:19.161587 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1401/2779] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:29:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:29:19.165682 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e67cab-959f-4a6f-8b1c-65e2e804f4e8 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:29:19.176104 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e67cab-959f-4a6f-8b1c-65e2e804f4e8 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:29:19.180669 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaef28cf-1216-43b2-a88b-e67db0fcd8bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:29:19.181561 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaef28cf-1216-43b2-a88b-e67db0fcd8bf admin admin] REQUEST_METHOD: `PUT` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:29:19.181845 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaef28cf-1216-43b2-a88b-e67db0fcd8bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:29:19.182126 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaef28cf-1216-43b2-a88b-e67db0fcd8bf admin admin] PATH_INFO: `/v3/projects/62211c4027dc4d7c9aae7143ea814468/users/644c34990c91432f80ed8b73d4bb1fde/roles/cbf5c4c26898413d93e73b46b41046a3` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:29:19.197578 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaef28cf-1216-43b2-a88b-e67db0fcd8bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=62211c4027dc4d7c9aae7143ea814468, user_id=644c34990c91432f80ed8b73d4bb1fde, role_id=cbf5c4c26898413d93e73b46b41046a3)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:29:19.200233 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaef28cf-1216-43b2-a88b-e67db0fcd8bf admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:29:19.269811 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1379/2780] 10.222.0.23 () {60 vars in 1482 bytes} [Sat Jun 18 05:29:19 2022] PUT /identity/v3/projects/62211c4027dc4d7c9aae7143ea814468/users/644c34990c91432f80ed8b73d4bb1fde/roles/cbf5c4c26898413d93e73b46b41046a3 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:29:19.280012 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc652726-fa54-4d06-8555-f0bddc8fc3e7 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:29:19.280304 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc652726-fa54-4d06-8555-f0bddc8fc3e7 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:29:19.280587 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc652726-fa54-4d06-8555-f0bddc8fc3e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:29:19.293884 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:29:19.308328 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-dc652726-fa54-4d06-8555-f0bddc8fc3e7 None None] MFA Rules not processed for user `644c34990c91432f80ed8b73d4bb1fde`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:29:19.309576 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-dc652726-fa54-4d06-8555-f0bddc8fc3e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:29:19.374347 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1402/2781] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:29:19 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:29:19.378969 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaef28cf-1216-43b2-a88b-e67db0fcd8bf admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:29:19.441875 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaef28cf-1216-43b2-a88b-e67db0fcd8bf admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:29:19.446567 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49aa0e1-a02a-4b79-8c4e-0953364fe2d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:29:19.447213 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49aa0e1-a02a-4b79-8c4e-0953364fe2d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:29:19.447499 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49aa0e1-a02a-4b79-8c4e-0953364fe2d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:29:19.447776 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49aa0e1-a02a-4b79-8c4e-0953364fe2d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:29:19.448472 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49aa0e1-a02a-4b79-8c4e-0953364fe2d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:29:19.450422 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49aa0e1-a02a-4b79-8c4e-0953364fe2d2 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:29:19.557117 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1380/2782] 10.222.0.23 () {60 vars in 1152 bytes} [Sat Jun 18 05:29:19 2022] POST /identity/v3/projects => generated 387 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:29:19.561086 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc652726-fa54-4d06-8555-f0bddc8fc3e7 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:29:19.572143 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc652726-fa54-4d06-8555-f0bddc8fc3e7 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:29:19.628332 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3cd874-0d58-4bf9-a253-aafeea73b441 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:29:19.628927 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3cd874-0d58-4bf9-a253-aafeea73b441 admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:29:19.629210 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3cd874-0d58-4bf9-a253-aafeea73b441 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:29:19.629516 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3cd874-0d58-4bf9-a253-aafeea73b441 admin admin] PATH_INFO: `/v3/users` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:29:19.630209 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3cd874-0d58-4bf9-a253-aafeea73b441 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:29:19.632172 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3cd874-0d58-4bf9-a253-aafeea73b441 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:29:19.707618 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1403/2783] 10.222.0.23 () {60 vars in 1143 bytes} [Sat Jun 18 05:29:19 2022] POST /identity/v3/users => generated 294 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 18 05:29:19.711575 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49aa0e1-a02a-4b79-8c4e-0953364fe2d2 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:29:19.722415 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49aa0e1-a02a-4b79-8c4e-0953364fe2d2 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:29:19.727061 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6fd3c1-8b06-4239-b4f4-eade70a0c8b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:29:19.727706 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6fd3c1-8b06-4239-b4f4-eade70a0c8b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:29:19.730342 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6fd3c1-8b06-4239-b4f4-eade70a0c8b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:29:19.730342 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6fd3c1-8b06-4239-b4f4-eade70a0c8b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:29:19.730342 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6fd3c1-8b06-4239-b4f4-eade70a0c8b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:29:19.730718 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6fd3c1-8b06-4239-b4f4-eade70a0c8b7 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:29:19.740908 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1381/2784] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:29:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:29:19.745080 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3cd874-0d58-4bf9-a253-aafeea73b441 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:29:19.755447 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3cd874-0d58-4bf9-a253-aafeea73b441 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:29:19.760066 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0860aff-6c98-4154-8161-5a36bf9b4965 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:29:19.760710 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0860aff-6c98-4154-8161-5a36bf9b4965 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:29:19.761007 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0860aff-6c98-4154-8161-5a36bf9b4965 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:29:19.761265 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0860aff-6c98-4154-8161-5a36bf9b4965 admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:29:19.762024 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0860aff-6c98-4154-8161-5a36bf9b4965 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:29:19.763693 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0860aff-6c98-4154-8161-5a36bf9b4965 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:29:19.774269 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1404/2785] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:29:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:29:19.778226 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6fd3c1-8b06-4239-b4f4-eade70a0c8b7 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:29:19.789164 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6fd3c1-8b06-4239-b4f4-eade70a0c8b7 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:29:19.793758 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252f36d5-3393-46d4-a911-2b8663aec57c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:29:19.794614 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252f36d5-3393-46d4-a911-2b8663aec57c admin admin] REQUEST_METHOD: `PUT` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:29:19.794897 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252f36d5-3393-46d4-a911-2b8663aec57c admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:29:19.795173 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252f36d5-3393-46d4-a911-2b8663aec57c admin admin] PATH_INFO: `/v3/projects/63a48876a6e04028b448ed616e71bdca/users/c96adc584fcd4315a49395c651aa5fdf/roles/cbf5c4c26898413d93e73b46b41046a3` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:29:19.809397 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252f36d5-3393-46d4-a911-2b8663aec57c admin admin] RBAC: Authorizing `identity:create_grant(project_id=63a48876a6e04028b448ed616e71bdca, user_id=c96adc584fcd4315a49395c651aa5fdf, role_id=cbf5c4c26898413d93e73b46b41046a3)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:29:19.812044 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252f36d5-3393-46d4-a911-2b8663aec57c admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:29:19.876007 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1382/2786] 10.222.0.23 () {60 vars in 1482 bytes} [Sat Jun 18 05:29:19 2022] PUT /identity/v3/projects/63a48876a6e04028b448ed616e71bdca/users/c96adc584fcd4315a49395c651aa5fdf/roles/cbf5c4c26898413d93e73b46b41046a3 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:29:19.887755 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdbeb65-d56e-4f29-b2f9-1133b028650e None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:29:19.887979 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdbeb65-d56e-4f29-b2f9-1133b028650e None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:29:19.888257 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdbeb65-d56e-4f29-b2f9-1133b028650e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:29:19.915908 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-ccdbeb65-d56e-4f29-b2f9-1133b028650e None None] MFA Rules not processed for user `c96adc584fcd4315a49395c651aa5fdf`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:29:19.917111 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-ccdbeb65-d56e-4f29-b2f9-1133b028650e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:29:19.979993 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1405/2787] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:29:19 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:29:19.984473 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252f36d5-3393-46d4-a911-2b8663aec57c admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:29:19.997366 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:29:20.048367 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252f36d5-3393-46d4-a911-2b8663aec57c admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:29:20.053608 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033414e0-e39c-4942-8c47-f525300f5995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:29:20.054256 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033414e0-e39c-4942-8c47-f525300f5995 admin admin] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:29:20.054542 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033414e0-e39c-4942-8c47-f525300f5995 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:29:20.054848 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033414e0-e39c-4942-8c47-f525300f5995 admin admin] PATH_INFO: `/v3/projects` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:29:20.055522 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033414e0-e39c-4942-8c47-f525300f5995 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:29:20.057455 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033414e0-e39c-4942-8c47-f525300f5995 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:29:20.139529 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1383/2788] 10.222.0.23 () {60 vars in 1152 bytes} [Sat Jun 18 05:29:19 2022] POST /identity/v3/projects => generated 385 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:29:20.144140 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdbeb65-d56e-4f29-b2f9-1133b028650e None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:29:20.150833 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:29:20.154985 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdbeb65-d56e-4f29-b2f9-1133b028650e None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:29:20.211613 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcbca2a-84be-4ef6-85dc-4f278e1ccdd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:29:20.212281 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcbca2a-84be-4ef6-85dc-4f278e1ccdd6 admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:29:20.212566 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcbca2a-84be-4ef6-85dc-4f278e1ccdd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:29:20.212842 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcbca2a-84be-4ef6-85dc-4f278e1ccdd6 admin admin] PATH_INFO: `/v3/users` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:29:20.213559 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcbca2a-84be-4ef6-85dc-4f278e1ccdd6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:29:20.215628 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcbca2a-84be-4ef6-85dc-4f278e1ccdd6 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:29:20.319521 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1406/2789] 10.222.0.23 () {60 vars in 1143 bytes} [Sat Jun 18 05:29:20 2022] POST /identity/v3/users => generated 299 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:29:20.323606 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033414e0-e39c-4942-8c47-f525300f5995 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:29:20.334605 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033414e0-e39c-4942-8c47-f525300f5995 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:29:20.339209 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c7147b-5ac2-45c8-a85a-3dc6cafbbd60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:29:20.339847 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c7147b-5ac2-45c8-a85a-3dc6cafbbd60 admin admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:29:20.340126 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c7147b-5ac2-45c8-a85a-3dc6cafbbd60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:29:20.340450 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c7147b-5ac2-45c8-a85a-3dc6cafbbd60 admin admin] PATH_INFO: `/v3/roles` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:29:20.341173 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c7147b-5ac2-45c8-a85a-3dc6cafbbd60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:29:20.342793 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c7147b-5ac2-45c8-a85a-3dc6cafbbd60 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:29:20.352649 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1384/2790] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:29:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:29:20.356798 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcbca2a-84be-4ef6-85dc-4f278e1ccdd6 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:29:20.367292 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcbca2a-84be-4ef6-85dc-4f278e1ccdd6 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:29:20.372002 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235a92dd-0ec2-4ffc-b3cd-69bc12948ccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:29:20.372867 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235a92dd-0ec2-4ffc-b3cd-69bc12948ccc admin admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:29:20.373151 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235a92dd-0ec2-4ffc-b3cd-69bc12948ccc admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:29:20.373484 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235a92dd-0ec2-4ffc-b3cd-69bc12948ccc admin admin] PATH_INFO: `/v3/projects/134625a67b5f4c76a65554094e540926/users/bc76b33d5887407780b6c827be13fc21/roles/21c65f2c51bd4dd283d13507d2714467` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:29:20.389284 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235a92dd-0ec2-4ffc-b3cd-69bc12948ccc admin admin] RBAC: Authorizing `identity:create_grant(project_id=134625a67b5f4c76a65554094e540926, user_id=bc76b33d5887407780b6c827be13fc21, role_id=21c65f2c51bd4dd283d13507d2714467)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:29:20.392777 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235a92dd-0ec2-4ffc-b3cd-69bc12948ccc admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:29:20.462107 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1407/2791] 10.222.0.23 () {60 vars in 1482 bytes} [Sat Jun 18 05:29:20 2022] PUT /identity/v3/projects/134625a67b5f4c76a65554094e540926/users/bc76b33d5887407780b6c827be13fc21/roles/21c65f2c51bd4dd283d13507d2714467 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:29:20.466088 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c7147b-5ac2-45c8-a85a-3dc6cafbbd60 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:29:20.476243 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c7147b-5ac2-45c8-a85a-3dc6cafbbd60 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:29:20.531579 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662a539b-f1fd-4447-93d3-a34a1dc8c2ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:29:20.532233 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662a539b-f1fd-4447-93d3-a34a1dc8c2ae admin admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:29:20.532511 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662a539b-f1fd-4447-93d3-a34a1dc8c2ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:29:20.532783 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662a539b-f1fd-4447-93d3-a34a1dc8c2ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:29:20.533528 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662a539b-f1fd-4447-93d3-a34a1dc8c2ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:29:20.535352 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662a539b-f1fd-4447-93d3-a34a1dc8c2ae admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:29:20.544807 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1385/2792] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:29:20 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:29:20.550157 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235a92dd-0ec2-4ffc-b3cd-69bc12948ccc admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:29:20.560553 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235a92dd-0ec2-4ffc-b3cd-69bc12948ccc admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:29:20.565993 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce98ad2-5ad1-4bc0-b2ee-661f7797e424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:29:20.566892 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce98ad2-5ad1-4bc0-b2ee-661f7797e424 admin admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:29:20.567173 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce98ad2-5ad1-4bc0-b2ee-661f7797e424 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:29:20.567450 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce98ad2-5ad1-4bc0-b2ee-661f7797e424 admin admin] PATH_INFO: `/v3/projects/134625a67b5f4c76a65554094e540926/users/bc76b33d5887407780b6c827be13fc21/roles/21c65f2c51bd4dd283d13507d2714467` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:29:20.569785 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce98ad2-5ad1-4bc0-b2ee-661f7797e424 admin admin] RBAC: Authorizing `identity:create_grant(project_id=134625a67b5f4c76a65554094e540926, user_id=bc76b33d5887407780b6c827be13fc21, role_id=21c65f2c51bd4dd283d13507d2714467)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:29:20.572410 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce98ad2-5ad1-4bc0-b2ee-661f7797e424 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:29:20.583275 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1408/2793] 10.222.0.23 () {60 vars in 1482 bytes} [Sat Jun 18 05:29:20 2022] PUT /identity/v3/projects/134625a67b5f4c76a65554094e540926/users/bc76b33d5887407780b6c827be13fc21/roles/21c65f2c51bd4dd283d13507d2714467 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:29:20.593336 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a558f4-f903-40eb-8c31-af166e85ee3b None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:29:20.593655 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a558f4-f903-40eb-8c31-af166e85ee3b None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:29:20.593929 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a558f4-f903-40eb-8c31-af166e85ee3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:29:20.620384 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-a5a558f4-f903-40eb-8c31-af166e85ee3b None None] MFA Rules not processed for user `bc76b33d5887407780b6c827be13fc21`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:29:20.621609 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-a5a558f4-f903-40eb-8c31-af166e85ee3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:29:20.689172 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1386/2794] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:29:20 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:29:20.699576 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce98ad2-5ad1-4bc0-b2ee-661f7797e424 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:29:20.701260 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-1ce98ad2-5ad1-4bc0-b2ee-661f7797e424 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:29:20.759377 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce98ad2-5ad1-4bc0-b2ee-661f7797e424 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:29:20.763386 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1025a8a4-e56d-4827-a594-3dbac66a4a26 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9de0428abc8400aa338aba4f7c34c94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:29:20.764026 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1025a8a4-e56d-4827-a594-3dbac66a4a26 service cinder] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:29:20.764311 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1025a8a4-e56d-4827-a594-3dbac66a4a26 service cinder] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:29:20.764588 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1025a8a4-e56d-4827-a594-3dbac66a4a26 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:29:20.773926 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1025a8a4-e56d-4827-a594-3dbac66a4a26 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:29:20.775373 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1025a8a4-e56d-4827-a594-3dbac66a4a26 service cinder] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:29:20.786108 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1409/2795] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:29:20 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:29:21.971099 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a558f4-f903-40eb-8c31-af166e85ee3b None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:29:21.972545 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-a5a558f4-f903-40eb-8c31-af166e85ee3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:29:21.984569 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a558f4-f903-40eb-8c31-af166e85ee3b None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:29:21.988657 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7529848d-6505-4e04-a2fd-4161e7997d63 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9de0428abc8400aa338aba4f7c34c94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:29:21.989342 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7529848d-6505-4e04-a2fd-4161e7997d63 service cinder] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:29:21.989665 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7529848d-6505-4e04-a2fd-4161e7997d63 service cinder] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:29:21.989946 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7529848d-6505-4e04-a2fd-4161e7997d63 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:29:21.994725 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:29:22.049333 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7529848d-6505-4e04-a2fd-4161e7997d63 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:29:22.050791 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7529848d-6505-4e04-a2fd-4161e7997d63 service cinder] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:29:22.059808 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1387/2796] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:29:21 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:29:26.836971 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1025a8a4-e56d-4827-a594-3dbac66a4a26 service cinder] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:29:26.846902 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1025a8a4-e56d-4827-a594-3dbac66a4a26 service cinder] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:29:26.851020 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83909fa-4a23-438f-83b6-ebf74fff5c52 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9de0428abc8400aa338aba4f7c34c94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:29:26.856108 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83909fa-4a23-438f-83b6-ebf74fff5c52 service cinder] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:29:26.856108 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83909fa-4a23-438f-83b6-ebf74fff5c52 service cinder] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:29:26.856108 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83909fa-4a23-438f-83b6-ebf74fff5c52 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:29:26.911855 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83909fa-4a23-438f-83b6-ebf74fff5c52 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:29:26.913214 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83909fa-4a23-438f-83b6-ebf74fff5c52 service cinder] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:29:26.916858 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:29:26.923338 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1410/2797] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:29:26 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:29:30.603624 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7529848d-6505-4e04-a2fd-4161e7997d63 service cinder] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:29:30.666043 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7529848d-6505-4e04-a2fd-4161e7997d63 service cinder] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:29:30.670484 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809d2dcc-b619-46eb-a7fa-38a471787836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:29:30.671330 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809d2dcc-b619-46eb-a7fa-38a471787836 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:29:30.671613 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809d2dcc-b619-46eb-a7fa-38a471787836 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:29:30.671889 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809d2dcc-b619-46eb-a7fa-38a471787836 admin admin] PATH_INFO: `/v3/users/644c34990c91432f80ed8b73d4bb1fde` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:29:30.673069 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809d2dcc-b619-46eb-a7fa-38a471787836 admin admin] RBAC: Authorizing `identity:delete_user(user_id=644c34990c91432f80ed8b73d4bb1fde)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:29:30.675017 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809d2dcc-b619-46eb-a7fa-38a471787836 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:29:30.845859 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-809d2dcc-b619-46eb-a7fa-38a471787836 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '644c34990c91432f80ed8b73d4bb1fde'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:29:30.846507 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-809d2dcc-b619-46eb-a7fa-38a471787836 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '644c34990c91432f80ed8b73d4bb1fde'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:29:30.918380 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-809d2dcc-b619-46eb-a7fa-38a471787836 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '644c34990c91432f80ed8b73d4bb1fde'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:29:30.931321 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-809d2dcc-b619-46eb-a7fa-38a471787836 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '644c34990c91432f80ed8b73d4bb1fde'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:29:30.938219 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-809d2dcc-b619-46eb-a7fa-38a471787836 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '644c34990c91432f80ed8b73d4bb1fde'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:29:30.944975 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1388/2798] 10.222.0.23 () {58 vars in 1223 bytes} [Sat Jun 18 05:29:30 2022] DELETE /identity/v3/users/644c34990c91432f80ed8b73d4bb1fde => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:29:30.949503 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ee5d2f-b300-40b5-b0a3-fcd78f998248 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:29:30.949992 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ee5d2f-b300-40b5-b0a3-fcd78f998248 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:29:30.950117 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ee5d2f-b300-40b5-b0a3-fcd78f998248 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:29:30.967103 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-d6ee5d2f-b300-40b5-b0a3-fcd78f998248 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:29:30.968280 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-d6ee5d2f-b300-40b5-b0a3-fcd78f998248 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:29:31.028276 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1411/2799] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:29:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:29:31.038257 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809d2dcc-b619-46eb-a7fa-38a471787836 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:29:31.039673 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-809d2dcc-b619-46eb-a7fa-38a471787836 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:29:31.068236 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:29:31.096974 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809d2dcc-b619-46eb-a7fa-38a471787836 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:29:31.101272 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53828824-272a-414f-9d88-382d776b1efe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c4d7cde8f4ad9aa479bb09e325db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:29:31.101948 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53828824-272a-414f-9d88-382d776b1efe service neutron] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:29:31.102233 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53828824-272a-414f-9d88-382d776b1efe service neutron] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:29:31.105476 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53828824-272a-414f-9d88-382d776b1efe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:29:31.111632 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53828824-272a-414f-9d88-382d776b1efe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:29:31.113054 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53828824-272a-414f-9d88-382d776b1efe service neutron] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:29:31.123769 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1389/2800] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:29:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:29:31.505691 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ee5d2f-b300-40b5-b0a3-fcd78f998248 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:29:31.507198 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-d6ee5d2f-b300-40b5-b0a3-fcd78f998248 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:29:31.520856 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ee5d2f-b300-40b5-b0a3-fcd78f998248 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:29:31.525593 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad395e04-4210-43ce-beb6-fc949b096766 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:29:31.526460 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad395e04-4210-43ce-beb6-fc949b096766 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:29:31.526747 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad395e04-4210-43ce-beb6-fc949b096766 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:29:31.527026 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad395e04-4210-43ce-beb6-fc949b096766 admin admin] PATH_INFO: `/v3/projects/62211c4027dc4d7c9aae7143ea814468` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:29:31.528228 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad395e04-4210-43ce-beb6-fc949b096766 admin admin] RBAC: Authorizing `identity:delete_project(project_id=62211c4027dc4d7c9aae7143ea814468)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:29:31.530226 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad395e04-4210-43ce-beb6-fc949b096766 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:29:31.641505 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-ad395e04-4210-43ce-beb6-fc949b096766 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '62211c4027dc4d7c9aae7143ea814468', 'request_id': 'req-ad395e04-4210-43ce-beb6-fc949b096766'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:29:31.647576 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-ad395e04-4210-43ce-beb6-fc949b096766 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '62211c4027dc4d7c9aae7143ea814468', 'request_id': 'req-ad395e04-4210-43ce-beb6-fc949b096766'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:29:31.654010 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-ad395e04-4210-43ce-beb6-fc949b096766 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '62211c4027dc4d7c9aae7143ea814468', 'request_id': 'req-ad395e04-4210-43ce-beb6-fc949b096766'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:29:31.731949 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-ad395e04-4210-43ce-beb6-fc949b096766 admin admin] The token cache is being invalidate because project 62211c4027dc4d7c9aae7143ea814468 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80469) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 18 05:29:31.732238 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-ad395e04-4210-43ce-beb6-fc949b096766 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:29:31.732902 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-ad395e04-4210-43ce-beb6-fc949b096766 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:29:31.734859 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1412/2801] 10.222.0.23 () {58 vars in 1232 bytes} [Sat Jun 18 05:29:31 2022] DELETE /identity/v3/projects/62211c4027dc4d7c9aae7143ea814468 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:29:31.739044 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53828824-272a-414f-9d88-382d776b1efe service neutron] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:29:31.748774 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53828824-272a-414f-9d88-382d776b1efe service neutron] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:29:31.804623 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d5a573-f396-415f-a2dd-c41347860284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:29:31.805510 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d5a573-f396-415f-a2dd-c41347860284 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:29:31.812118 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d5a573-f396-415f-a2dd-c41347860284 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:29:31.812118 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d5a573-f396-415f-a2dd-c41347860284 admin admin] PATH_INFO: `/v3/users/c96adc584fcd4315a49395c651aa5fdf` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:29:31.812118 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d5a573-f396-415f-a2dd-c41347860284 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c96adc584fcd4315a49395c651aa5fdf)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:29:31.812118 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d5a573-f396-415f-a2dd-c41347860284 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:29:31.989161 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-d0d5a573-f396-415f-a2dd-c41347860284 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c96adc584fcd4315a49395c651aa5fdf'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:29:31.989848 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-d0d5a573-f396-415f-a2dd-c41347860284 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c96adc584fcd4315a49395c651aa5fdf'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:29:32.044149 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-d0d5a573-f396-415f-a2dd-c41347860284 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c96adc584fcd4315a49395c651aa5fdf'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:29:32.056795 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-d0d5a573-f396-415f-a2dd-c41347860284 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c96adc584fcd4315a49395c651aa5fdf'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:29:32.063654 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-d0d5a573-f396-415f-a2dd-c41347860284 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c96adc584fcd4315a49395c651aa5fdf'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:29:32.070146 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1390/2802] 10.222.0.23 () {58 vars in 1223 bytes} [Sat Jun 18 05:29:31 2022] DELETE /identity/v3/users/c96adc584fcd4315a49395c651aa5fdf => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:29:32.454045 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad395e04-4210-43ce-beb6-fc949b096766 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:29:32.455824 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-ad395e04-4210-43ce-beb6-fc949b096766 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:29:32.470920 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad395e04-4210-43ce-beb6-fc949b096766 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:29:32.528734 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddb7e53-e02e-440b-ad76-678149bffc43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:29:32.529646 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddb7e53-e02e-440b-ad76-678149bffc43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:29:32.529933 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddb7e53-e02e-440b-ad76-678149bffc43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:29:32.530239 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddb7e53-e02e-440b-ad76-678149bffc43 admin admin] PATH_INFO: `/v3/projects/63a48876a6e04028b448ed616e71bdca` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:29:32.531441 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddb7e53-e02e-440b-ad76-678149bffc43 admin admin] RBAC: Authorizing `identity:delete_project(project_id=63a48876a6e04028b448ed616e71bdca)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:29:32.533410 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddb7e53-e02e-440b-ad76-678149bffc43 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:29:32.558904 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:29:32.658038 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-fddb7e53-e02e-440b-ad76-678149bffc43 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63a48876a6e04028b448ed616e71bdca', 'request_id': 'req-fddb7e53-e02e-440b-ad76-678149bffc43'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:29:32.664600 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-fddb7e53-e02e-440b-ad76-678149bffc43 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63a48876a6e04028b448ed616e71bdca', 'request_id': 'req-fddb7e53-e02e-440b-ad76-678149bffc43'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:29:32.670905 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-fddb7e53-e02e-440b-ad76-678149bffc43 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63a48876a6e04028b448ed616e71bdca', 'request_id': 'req-fddb7e53-e02e-440b-ad76-678149bffc43'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:29:32.783288 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-fddb7e53-e02e-440b-ad76-678149bffc43 admin admin] The token cache is being invalidate because project 63a48876a6e04028b448ed616e71bdca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80469) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 18 05:29:32.783575 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-fddb7e53-e02e-440b-ad76-678149bffc43 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:29:32.784461 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-fddb7e53-e02e-440b-ad76-678149bffc43 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:29:32.786639 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1413/2803] 10.222.0.23 () {58 vars in 1232 bytes} [Sat Jun 18 05:29:32 2022] DELETE /identity/v3/projects/63a48876a6e04028b448ed616e71bdca => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:29:32.790512 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d5a573-f396-415f-a2dd-c41347860284 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:29:32.791943 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-d0d5a573-f396-415f-a2dd-c41347860284 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:29:32.805285 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d5a573-f396-415f-a2dd-c41347860284 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:29:32.849328 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:29:32.861843 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811cf154-bdd3-4788-ace4-831c2f8b95bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:29:32.862792 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811cf154-bdd3-4788-ace4-831c2f8b95bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:29:32.863191 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811cf154-bdd3-4788-ace4-831c2f8b95bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:29:32.863468 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811cf154-bdd3-4788-ace4-831c2f8b95bf admin admin] PATH_INFO: `/v3/users/bc76b33d5887407780b6c827be13fc21` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:29:32.865740 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811cf154-bdd3-4788-ace4-831c2f8b95bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc76b33d5887407780b6c827be13fc21)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:29:32.867642 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811cf154-bdd3-4788-ace4-831c2f8b95bf admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:29:33.075561 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-811cf154-bdd3-4788-ace4-831c2f8b95bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc76b33d5887407780b6c827be13fc21'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:29:33.076220 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-811cf154-bdd3-4788-ace4-831c2f8b95bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc76b33d5887407780b6c827be13fc21'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:29:33.138480 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-811cf154-bdd3-4788-ace4-831c2f8b95bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc76b33d5887407780b6c827be13fc21'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:29:33.152643 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-811cf154-bdd3-4788-ace4-831c2f8b95bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc76b33d5887407780b6c827be13fc21'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:29:33.160111 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-811cf154-bdd3-4788-ace4-831c2f8b95bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc76b33d5887407780b6c827be13fc21'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:29:33.167329 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1391/2804] 10.222.0.23 () {58 vars in 1223 bytes} [Sat Jun 18 05:29:32 2022] DELETE /identity/v3/users/bc76b33d5887407780b6c827be13fc21 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:29:33.600711 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddb7e53-e02e-440b-ad76-678149bffc43 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:29:33.602242 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-fddb7e53-e02e-440b-ad76-678149bffc43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:29:33.617601 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddb7e53-e02e-440b-ad76-678149bffc43 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:29:33.678063 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8ca755-e9ab-46a3-9131-bd2348abaa13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:29:33.678934 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8ca755-e9ab-46a3-9131-bd2348abaa13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:29:33.679224 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8ca755-e9ab-46a3-9131-bd2348abaa13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:29:33.679503 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8ca755-e9ab-46a3-9131-bd2348abaa13 admin admin] PATH_INFO: `/v3/projects/134625a67b5f4c76a65554094e540926` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:29:33.680711 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8ca755-e9ab-46a3-9131-bd2348abaa13 admin admin] RBAC: Authorizing `identity:delete_project(project_id=134625a67b5f4c76a65554094e540926)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:29:33.682662 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8ca755-e9ab-46a3-9131-bd2348abaa13 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:29:33.795993 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-ba8ca755-e9ab-46a3-9131-bd2348abaa13 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '134625a67b5f4c76a65554094e540926', 'request_id': 'req-ba8ca755-e9ab-46a3-9131-bd2348abaa13'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:29:33.801704 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-ba8ca755-e9ab-46a3-9131-bd2348abaa13 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '134625a67b5f4c76a65554094e540926', 'request_id': 'req-ba8ca755-e9ab-46a3-9131-bd2348abaa13'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:29:33.809081 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-ba8ca755-e9ab-46a3-9131-bd2348abaa13 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '134625a67b5f4c76a65554094e540926', 'request_id': 'req-ba8ca755-e9ab-46a3-9131-bd2348abaa13'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:29:33.880898 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-ba8ca755-e9ab-46a3-9131-bd2348abaa13 admin admin] The token cache is being invalidate because project 134625a67b5f4c76a65554094e540926 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80469) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 18 05:29:33.881129 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-ba8ca755-e9ab-46a3-9131-bd2348abaa13 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:29:33.881788 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-ba8ca755-e9ab-46a3-9131-bd2348abaa13 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:29:33.883427 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1414/2805] 10.222.0.23 () {58 vars in 1232 bytes} [Sat Jun 18 05:29:33 2022] DELETE /identity/v3/projects/134625a67b5f4c76a65554094e540926 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:29:33.889447 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5501fdac-ae5f-4b18-8cfd-5c33dd970c5e None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:29:33.889973 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5501fdac-ae5f-4b18-8cfd-5c33dd970c5e None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:29:33.890093 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5501fdac-ae5f-4b18-8cfd-5c33dd970c5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:29:33.910691 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-5501fdac-ae5f-4b18-8cfd-5c33dd970c5e None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:29:33.912486 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-5501fdac-ae5f-4b18-8cfd-5c33dd970c5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:29:33.972472 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1392/2806] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:29:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:29:33.981551 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba1ff60-3c59-483f-b275-2227b1914663 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:29:33.981842 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba1ff60-3c59-483f-b275-2227b1914663 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:29:33.982123 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba1ff60-3c59-483f-b275-2227b1914663 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:29:34.001962 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-cba1ff60-3c59-483f-b275-2227b1914663 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:29:34.003183 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-cba1ff60-3c59-483f-b275-2227b1914663 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:29:34.011237 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1415/2807] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:29:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:29:34.015429 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5501fdac-ae5f-4b18-8cfd-5c33dd970c5e None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:29:34.026507 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5501fdac-ae5f-4b18-8cfd-5c33dd970c5e None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:29:34.031261 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e8f1ae-2fa3-4324-97e5-329b0e8dcd94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:29:34.031935 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e8f1ae-2fa3-4324-97e5-329b0e8dcd94 admin admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:29:34.032219 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e8f1ae-2fa3-4324-97e5-329b0e8dcd94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:29:34.032494 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e8f1ae-2fa3-4324-97e5-329b0e8dcd94 admin admin] PATH_INFO: `/v3/domains` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:29:34.033239 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e8f1ae-2fa3-4324-97e5-329b0e8dcd94 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 05:29:34.033582 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e8f1ae-2fa3-4324-97e5-329b0e8dcd94 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:29:34.035428 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e8f1ae-2fa3-4324-97e5-329b0e8dcd94 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:29:34.048940 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1393/2808] 10.222.0.23 () {58 vars in 1152 bytes} [Sat Jun 18 05:29:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 05:29:34.053180 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba1ff60-3c59-483f-b275-2227b1914663 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:29:34.062488 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba1ff60-3c59-483f-b275-2227b1914663 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:29:34.067468 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c0811e-ffba-4ddf-af3f-126d47fb6ff5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:29:34.068125 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c0811e-ffba-4ddf-af3f-126d47fb6ff5 admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:29:34.068443 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c0811e-ffba-4ddf-af3f-126d47fb6ff5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:29:34.068720 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c0811e-ffba-4ddf-af3f-126d47fb6ff5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:29:34.069443 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c0811e-ffba-4ddf-af3f-126d47fb6ff5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:29:34.071361 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c0811e-ffba-4ddf-af3f-126d47fb6ff5 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:29:34.197882 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1416/2809] 10.222.0.23 () {60 vars in 1152 bytes} [Sat Jun 18 05:29:34 2022] POST /identity/v3/projects => generated 393 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:29:34.201776 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e8f1ae-2fa3-4324-97e5-329b0e8dcd94 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:29:34.212421 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e8f1ae-2fa3-4324-97e5-329b0e8dcd94 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:29:34.258956 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:29:34.267553 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcdda4e-82ef-4493-b760-8d256c9267b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:29:34.268207 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcdda4e-82ef-4493-b760-8d256c9267b0 admin admin] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:29:34.268487 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcdda4e-82ef-4493-b760-8d256c9267b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:29:34.268758 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcdda4e-82ef-4493-b760-8d256c9267b0 admin admin] PATH_INFO: `/v3/users` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:29:34.269502 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcdda4e-82ef-4493-b760-8d256c9267b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:29:34.272062 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcdda4e-82ef-4493-b760-8d256c9267b0 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:29:34.378351 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1394/2810] 10.222.0.23 () {60 vars in 1143 bytes} [Sat Jun 18 05:29:34 2022] POST /identity/v3/users => generated 297 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:29:34.383459 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c0811e-ffba-4ddf-af3f-126d47fb6ff5 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:29:34.394019 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c0811e-ffba-4ddf-af3f-126d47fb6ff5 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:29:34.399831 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5dd00f-ab49-494e-95de-c85ad46135d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:29:34.400488 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5dd00f-ab49-494e-95de-c85ad46135d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:29:34.400768 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5dd00f-ab49-494e-95de-c85ad46135d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:29:34.401163 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5dd00f-ab49-494e-95de-c85ad46135d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:29:34.401962 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5dd00f-ab49-494e-95de-c85ad46135d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:29:34.403558 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5dd00f-ab49-494e-95de-c85ad46135d0 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:29:34.414616 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1417/2811] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:29:34 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:29:34.418575 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcdda4e-82ef-4493-b760-8d256c9267b0 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:29:34.428731 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcdda4e-82ef-4493-b760-8d256c9267b0 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:29:34.433910 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c7a621-3994-45e6-86d1-c98c4385bcdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:29:34.434563 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c7a621-3994-45e6-86d1-c98c4385bcdc admin admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:29:34.434847 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c7a621-3994-45e6-86d1-c98c4385bcdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:29:34.435122 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c7a621-3994-45e6-86d1-c98c4385bcdc admin admin] PATH_INFO: `/v3/roles` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:29:34.435838 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c7a621-3994-45e6-86d1-c98c4385bcdc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:29:34.437474 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c7a621-3994-45e6-86d1-c98c4385bcdc admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:29:34.448755 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1395/2812] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:29:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:29:34.452740 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5dd00f-ab49-494e-95de-c85ad46135d0 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:29:34.458561 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:29:34.463986 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5dd00f-ab49-494e-95de-c85ad46135d0 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:29:34.468640 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8ace5d-660f-46f6-9d8b-7c091e9ca93c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:29:34.469542 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8ace5d-660f-46f6-9d8b-7c091e9ca93c admin admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:29:34.469823 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8ace5d-660f-46f6-9d8b-7c091e9ca93c admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:29:34.470094 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8ace5d-660f-46f6-9d8b-7c091e9ca93c admin admin] PATH_INFO: `/v3/projects/f29194a7cab84679b1ac9e9debee58e1/users/133cf8b9c0b245acb08a9247c82ac0a1/roles/cbf5c4c26898413d93e73b46b41046a3` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:29:34.484895 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8ace5d-660f-46f6-9d8b-7c091e9ca93c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f29194a7cab84679b1ac9e9debee58e1, user_id=133cf8b9c0b245acb08a9247c82ac0a1, role_id=cbf5c4c26898413d93e73b46b41046a3)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:29:34.487574 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8ace5d-660f-46f6-9d8b-7c091e9ca93c admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:29:34.575488 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1418/2813] 10.222.0.23 () {60 vars in 1482 bytes} [Sat Jun 18 05:29:34 2022] PUT /identity/v3/projects/f29194a7cab84679b1ac9e9debee58e1/users/133cf8b9c0b245acb08a9247c82ac0a1/roles/cbf5c4c26898413d93e73b46b41046a3 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 18 05:29:34.588074 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df82e79f-780b-44da-95dc-d6168de93e1b None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:29:34.588435 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df82e79f-780b-44da-95dc-d6168de93e1b None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:29:34.595255 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df82e79f-780b-44da-95dc-d6168de93e1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:29:34.615754 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-df82e79f-780b-44da-95dc-d6168de93e1b None None] MFA Rules not processed for user `133cf8b9c0b245acb08a9247c82ac0a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:29:34.616953 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-df82e79f-780b-44da-95dc-d6168de93e1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:29:34.682521 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1396/2814] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:29:34 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:29:34.686989 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8ace5d-660f-46f6-9d8b-7c091e9ca93c admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:29:34.748969 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8ace5d-660f-46f6-9d8b-7c091e9ca93c admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:29:34.753598 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd058a92-41f8-4807-9497-e2f44848c332 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:29:34.754239 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd058a92-41f8-4807-9497-e2f44848c332 admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:29:34.754568 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd058a92-41f8-4807-9497-e2f44848c332 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:29:34.755346 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd058a92-41f8-4807-9497-e2f44848c332 admin admin] PATH_INFO: `/v3/projects` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:29:34.756034 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd058a92-41f8-4807-9497-e2f44848c332 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:29:34.757960 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd058a92-41f8-4807-9497-e2f44848c332 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:29:34.874071 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1419/2815] 10.222.0.23 () {60 vars in 1152 bytes} [Sat Jun 18 05:29:34 2022] POST /identity/v3/projects => generated 391 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 18 05:29:34.878159 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df82e79f-780b-44da-95dc-d6168de93e1b None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:29:34.889554 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df82e79f-780b-44da-95dc-d6168de93e1b None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:29:34.946037 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3075181d-e535-4f50-8bd0-f8632a446070 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:29:34.946689 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3075181d-e535-4f50-8bd0-f8632a446070 admin admin] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:29:34.946973 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3075181d-e535-4f50-8bd0-f8632a446070 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:29:34.947249 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3075181d-e535-4f50-8bd0-f8632a446070 admin admin] PATH_INFO: `/v3/users` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:29:34.947925 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3075181d-e535-4f50-8bd0-f8632a446070 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:29:34.949859 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3075181d-e535-4f50-8bd0-f8632a446070 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:29:35.027849 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1397/2816] 10.222.0.23 () {60 vars in 1143 bytes} [Sat Jun 18 05:29:34 2022] POST /identity/v3/users => generated 302 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:29:35.031762 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd058a92-41f8-4807-9497-e2f44848c332 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:29:35.043318 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd058a92-41f8-4807-9497-e2f44848c332 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:29:35.047950 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1f03fe-1d44-4a58-8f8f-a77dd98f6015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:29:35.048588 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1f03fe-1d44-4a58-8f8f-a77dd98f6015 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:29:35.048868 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1f03fe-1d44-4a58-8f8f-a77dd98f6015 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:29:35.049140 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1f03fe-1d44-4a58-8f8f-a77dd98f6015 admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:29:35.054718 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1f03fe-1d44-4a58-8f8f-a77dd98f6015 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:29:35.054718 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1f03fe-1d44-4a58-8f8f-a77dd98f6015 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:29:35.062363 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1420/2817] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:29:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:29:35.066411 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3075181d-e535-4f50-8bd0-f8632a446070 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:29:35.077037 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3075181d-e535-4f50-8bd0-f8632a446070 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:29:35.081629 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e812d766-fa77-4f7b-bf56-609b8029c8be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:29:35.082494 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e812d766-fa77-4f7b-bf56-609b8029c8be admin admin] REQUEST_METHOD: `PUT` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:29:35.083375 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e812d766-fa77-4f7b-bf56-609b8029c8be admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:29:35.083655 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e812d766-fa77-4f7b-bf56-609b8029c8be admin admin] PATH_INFO: `/v3/projects/b4ba4c985f3344e3b0f640a1ec72b477/users/4ffe8131e54a47578e054a91bc5c5862/roles/21c65f2c51bd4dd283d13507d2714467` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:29:35.098406 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e812d766-fa77-4f7b-bf56-609b8029c8be admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4ba4c985f3344e3b0f640a1ec72b477, user_id=4ffe8131e54a47578e054a91bc5c5862, role_id=21c65f2c51bd4dd283d13507d2714467)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:29:35.101226 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e812d766-fa77-4f7b-bf56-609b8029c8be admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:29:35.152800 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1398/2818] 10.222.0.23 () {60 vars in 1482 bytes} [Sat Jun 18 05:29:35 2022] PUT /identity/v3/projects/b4ba4c985f3344e3b0f640a1ec72b477/users/4ffe8131e54a47578e054a91bc5c5862/roles/21c65f2c51bd4dd283d13507d2714467 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:29:35.156561 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1f03fe-1d44-4a58-8f8f-a77dd98f6015 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:29:35.167114 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1f03fe-1d44-4a58-8f8f-a77dd98f6015 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:29:35.223421 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5caffd1-073b-4aee-9659-efa37ba0c652 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:29:35.224170 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5caffd1-073b-4aee-9659-efa37ba0c652 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:29:35.224457 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5caffd1-073b-4aee-9659-efa37ba0c652 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:29:35.224738 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5caffd1-073b-4aee-9659-efa37ba0c652 admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:29:35.225491 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5caffd1-073b-4aee-9659-efa37ba0c652 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:29:35.227094 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5caffd1-073b-4aee-9659-efa37ba0c652 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:29:35.236831 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1421/2819] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:29:35 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:29:35.241851 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e812d766-fa77-4f7b-bf56-609b8029c8be admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:29:35.252331 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e812d766-fa77-4f7b-bf56-609b8029c8be admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:29:35.258272 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb30186-8d6c-40c4-8ee1-308a060f1720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:29:35.259131 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb30186-8d6c-40c4-8ee1-308a060f1720 admin admin] REQUEST_METHOD: `PUT` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:29:35.259409 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb30186-8d6c-40c4-8ee1-308a060f1720 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:29:35.259679 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb30186-8d6c-40c4-8ee1-308a060f1720 admin admin] PATH_INFO: `/v3/projects/b4ba4c985f3344e3b0f640a1ec72b477/users/4ffe8131e54a47578e054a91bc5c5862/roles/21c65f2c51bd4dd283d13507d2714467` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:29:35.261655 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb30186-8d6c-40c4-8ee1-308a060f1720 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4ba4c985f3344e3b0f640a1ec72b477, user_id=4ffe8131e54a47578e054a91bc5c5862, role_id=21c65f2c51bd4dd283d13507d2714467)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:29:35.264311 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb30186-8d6c-40c4-8ee1-308a060f1720 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:29:35.274590 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1399/2820] 10.222.0.23 () {60 vars in 1482 bytes} [Sat Jun 18 05:29:35 2022] PUT /identity/v3/projects/b4ba4c985f3344e3b0f640a1ec72b477/users/4ffe8131e54a47578e054a91bc5c5862/roles/21c65f2c51bd4dd283d13507d2714467 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:29:35.284665 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a5600e-d8c0-4702-b55b-dd04e0923115 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:29:35.284894 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a5600e-d8c0-4702-b55b-dd04e0923115 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:29:35.285171 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a5600e-d8c0-4702-b55b-dd04e0923115 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:29:35.312994 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-68a5600e-d8c0-4702-b55b-dd04e0923115 None None] MFA Rules not processed for user `4ffe8131e54a47578e054a91bc5c5862`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:29:35.314217 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-68a5600e-d8c0-4702-b55b-dd04e0923115 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:29:35.353758 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:29:35.385293 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1422/2821] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:29:35 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:29:35.401579 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb30186-8d6c-40c4-8ee1-308a060f1720 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:29:35.403349 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-ccb30186-8d6c-40c4-8ee1-308a060f1720 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:29:35.417947 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:29:35.461835 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb30186-8d6c-40c4-8ee1-308a060f1720 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:29:35.465940 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905c1b65-cf7c-49d2-adfc-01dc6ecdf476 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9de0428abc8400aa338aba4f7c34c94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:29:35.466576 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905c1b65-cf7c-49d2-adfc-01dc6ecdf476 service cinder] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:29:35.466860 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905c1b65-cf7c-49d2-adfc-01dc6ecdf476 service cinder] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:29:35.467135 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905c1b65-cf7c-49d2-adfc-01dc6ecdf476 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:29:35.476070 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905c1b65-cf7c-49d2-adfc-01dc6ecdf476 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:29:35.477524 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905c1b65-cf7c-49d2-adfc-01dc6ecdf476 service cinder] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:29:35.486845 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1400/2822] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:29:35 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:29:35.532504 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a5600e-d8c0-4702-b55b-dd04e0923115 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:29:35.533919 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-68a5600e-d8c0-4702-b55b-dd04e0923115 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:29:35.546933 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a5600e-d8c0-4702-b55b-dd04e0923115 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:29:35.550496 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06efb11e-0382-4ca5-acb1-e5bba3f0142a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9de0428abc8400aa338aba4f7c34c94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:29:35.551126 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06efb11e-0382-4ca5-acb1-e5bba3f0142a service cinder] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:29:35.551919 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06efb11e-0382-4ca5-acb1-e5bba3f0142a service cinder] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:29:35.552211 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06efb11e-0382-4ca5-acb1-e5bba3f0142a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:29:35.607252 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06efb11e-0382-4ca5-acb1-e5bba3f0142a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:29:35.608679 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06efb11e-0382-4ca5-acb1-e5bba3f0142a service cinder] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:29:35.618634 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1423/2823] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:29:35 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:29:40.054940 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905c1b65-cf7c-49d2-adfc-01dc6ecdf476 service cinder] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:29:40.115519 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905c1b65-cf7c-49d2-adfc-01dc6ecdf476 service cinder] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:29:40.120668 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a630b148-72b8-4c1a-9d11-f41afbcd5882 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:29:40.121553 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a630b148-72b8-4c1a-9d11-f41afbcd5882 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:29:40.121857 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a630b148-72b8-4c1a-9d11-f41afbcd5882 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:29:40.122127 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a630b148-72b8-4c1a-9d11-f41afbcd5882 admin admin] PATH_INFO: `/v3/users/133cf8b9c0b245acb08a9247c82ac0a1` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:29:40.123298 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a630b148-72b8-4c1a-9d11-f41afbcd5882 admin admin] RBAC: Authorizing `identity:delete_user(user_id=133cf8b9c0b245acb08a9247c82ac0a1)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:29:40.125190 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a630b148-72b8-4c1a-9d11-f41afbcd5882 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:29:40.506048 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-a630b148-72b8-4c1a-9d11-f41afbcd5882 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '133cf8b9c0b245acb08a9247c82ac0a1'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:29:40.506691 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-a630b148-72b8-4c1a-9d11-f41afbcd5882 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '133cf8b9c0b245acb08a9247c82ac0a1'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:29:40.601702 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-a630b148-72b8-4c1a-9d11-f41afbcd5882 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '133cf8b9c0b245acb08a9247c82ac0a1'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:29:40.614132 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-a630b148-72b8-4c1a-9d11-f41afbcd5882 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '133cf8b9c0b245acb08a9247c82ac0a1'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:29:40.621492 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-a630b148-72b8-4c1a-9d11-f41afbcd5882 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '133cf8b9c0b245acb08a9247c82ac0a1'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:29:40.627526 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1401/2824] 10.222.0.23 () {58 vars in 1223 bytes} [Sat Jun 18 05:29:40 2022] DELETE /identity/v3/users/133cf8b9c0b245acb08a9247c82ac0a1 => generated 0 bytes in 574 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:29:40.632560 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100f490f-5791-4733-9f06-13a693f3a224 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:29:40.633098 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100f490f-5791-4733-9f06-13a693f3a224 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:29:40.633229 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100f490f-5791-4733-9f06-13a693f3a224 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:29:40.650601 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-100f490f-5791-4733-9f06-13a693f3a224 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:29:40.651749 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-100f490f-5791-4733-9f06-13a693f3a224 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:29:40.709847 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1424/2825] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:29:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:29:40.719993 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a630b148-72b8-4c1a-9d11-f41afbcd5882 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:29:40.721398 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-a630b148-72b8-4c1a-9d11-f41afbcd5882 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:29:40.764593 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:29:40.782330 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a630b148-72b8-4c1a-9d11-f41afbcd5882 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:29:40.786447 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29c18a0-3976-4502-8882-30c9be09b160 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c4d7cde8f4ad9aa479bb09e325db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:29:40.787082 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29c18a0-3976-4502-8882-30c9be09b160 service neutron] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:29:40.787362 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29c18a0-3976-4502-8882-30c9be09b160 service neutron] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:29:40.787651 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29c18a0-3976-4502-8882-30c9be09b160 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:29:40.797545 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29c18a0-3976-4502-8882-30c9be09b160 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:29:40.799466 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29c18a0-3976-4502-8882-30c9be09b160 service neutron] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:29:40.810142 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1402/2826] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:29:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:29:41.249765 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100f490f-5791-4733-9f06-13a693f3a224 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:29:41.251138 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-100f490f-5791-4733-9f06-13a693f3a224 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:29:41.264195 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100f490f-5791-4733-9f06-13a693f3a224 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:29:41.269133 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff7331f-6bcd-4dcd-8245-bf2859e45c7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:29:41.270056 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff7331f-6bcd-4dcd-8245-bf2859e45c7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:29:41.279543 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff7331f-6bcd-4dcd-8245-bf2859e45c7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:29:41.279543 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff7331f-6bcd-4dcd-8245-bf2859e45c7e admin admin] PATH_INFO: `/v3/projects/f29194a7cab84679b1ac9e9debee58e1` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:29:41.279543 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff7331f-6bcd-4dcd-8245-bf2859e45c7e admin admin] RBAC: Authorizing `identity:delete_project(project_id=f29194a7cab84679b1ac9e9debee58e1)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:29:41.279543 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff7331f-6bcd-4dcd-8245-bf2859e45c7e admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:29:41.410961 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-cff7331f-6bcd-4dcd-8245-bf2859e45c7e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f29194a7cab84679b1ac9e9debee58e1', 'request_id': 'req-cff7331f-6bcd-4dcd-8245-bf2859e45c7e'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:29:41.416932 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-cff7331f-6bcd-4dcd-8245-bf2859e45c7e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f29194a7cab84679b1ac9e9debee58e1', 'request_id': 'req-cff7331f-6bcd-4dcd-8245-bf2859e45c7e'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:29:41.423143 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-cff7331f-6bcd-4dcd-8245-bf2859e45c7e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f29194a7cab84679b1ac9e9debee58e1', 'request_id': 'req-cff7331f-6bcd-4dcd-8245-bf2859e45c7e'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:29:41.426874 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:29:41.491789 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-cff7331f-6bcd-4dcd-8245-bf2859e45c7e admin admin] The token cache is being invalidate because project f29194a7cab84679b1ac9e9debee58e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80469) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 18 05:29:41.492154 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-cff7331f-6bcd-4dcd-8245-bf2859e45c7e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:29:41.492827 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-cff7331f-6bcd-4dcd-8245-bf2859e45c7e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:29:41.494488 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1425/2827] 10.222.0.23 () {58 vars in 1232 bytes} [Sat Jun 18 05:29:41 2022] DELETE /identity/v3/projects/f29194a7cab84679b1ac9e9debee58e1 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:29:41.498494 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29c18a0-3976-4502-8882-30c9be09b160 service neutron] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:29:41.509416 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29c18a0-3976-4502-8882-30c9be09b160 service neutron] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:29:41.567157 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06a4994-2101-4e57-a2f3-cec4685f41e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:29:41.568010 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06a4994-2101-4e57-a2f3-cec4685f41e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:29:41.568291 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06a4994-2101-4e57-a2f3-cec4685f41e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:29:41.568566 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06a4994-2101-4e57-a2f3-cec4685f41e1 admin admin] PATH_INFO: `/v3/users/4ffe8131e54a47578e054a91bc5c5862` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:29:41.570527 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06a4994-2101-4e57-a2f3-cec4685f41e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ffe8131e54a47578e054a91bc5c5862)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:29:41.572623 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06a4994-2101-4e57-a2f3-cec4685f41e1 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:29:41.800846 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-a06a4994-2101-4e57-a2f3-cec4685f41e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ffe8131e54a47578e054a91bc5c5862'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:29:41.801500 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-a06a4994-2101-4e57-a2f3-cec4685f41e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ffe8131e54a47578e054a91bc5c5862'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:29:41.873042 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-a06a4994-2101-4e57-a2f3-cec4685f41e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ffe8131e54a47578e054a91bc5c5862'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:29:41.885681 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-a06a4994-2101-4e57-a2f3-cec4685f41e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ffe8131e54a47578e054a91bc5c5862'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:29:41.892460 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-a06a4994-2101-4e57-a2f3-cec4685f41e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ffe8131e54a47578e054a91bc5c5862'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:29:41.898437 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1403/2828] 10.222.0.23 () {58 vars in 1223 bytes} [Sat Jun 18 05:29:41 2022] DELETE /identity/v3/users/4ffe8131e54a47578e054a91bc5c5862 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:29:42.314636 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff7331f-6bcd-4dcd-8245-bf2859e45c7e admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:29:42.316935 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-cff7331f-6bcd-4dcd-8245-bf2859e45c7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:29:42.330339 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff7331f-6bcd-4dcd-8245-bf2859e45c7e admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:29:42.388564 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f875fd-5948-47a8-b5cc-41ce82e0ed42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:29:42.389488 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f875fd-5948-47a8-b5cc-41ce82e0ed42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:29:42.389771 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f875fd-5948-47a8-b5cc-41ce82e0ed42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:29:42.390047 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f875fd-5948-47a8-b5cc-41ce82e0ed42 admin admin] PATH_INFO: `/v3/projects/b4ba4c985f3344e3b0f640a1ec72b477` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:29:42.391246 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f875fd-5948-47a8-b5cc-41ce82e0ed42 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b4ba4c985f3344e3b0f640a1ec72b477)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:29:42.393162 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f875fd-5948-47a8-b5cc-41ce82e0ed42 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:29:42.501940 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-b6f875fd-5948-47a8-b5cc-41ce82e0ed42 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b4ba4c985f3344e3b0f640a1ec72b477', 'request_id': 'req-b6f875fd-5948-47a8-b5cc-41ce82e0ed42'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:29:42.507918 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-b6f875fd-5948-47a8-b5cc-41ce82e0ed42 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b4ba4c985f3344e3b0f640a1ec72b477', 'request_id': 'req-b6f875fd-5948-47a8-b5cc-41ce82e0ed42'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:29:42.514283 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-b6f875fd-5948-47a8-b5cc-41ce82e0ed42 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b4ba4c985f3344e3b0f640a1ec72b477', 'request_id': 'req-b6f875fd-5948-47a8-b5cc-41ce82e0ed42'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:29:42.586936 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-b6f875fd-5948-47a8-b5cc-41ce82e0ed42 admin admin] The token cache is being invalidate because project b4ba4c985f3344e3b0f640a1ec72b477 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80469) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 18 05:29:42.587130 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-b6f875fd-5948-47a8-b5cc-41ce82e0ed42 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:29:42.587744 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-b6f875fd-5948-47a8-b5cc-41ce82e0ed42 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:29:42.589277 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1426/2829] 10.222.0.23 () {58 vars in 1232 bytes} [Sat Jun 18 05:29:42 2022] DELETE /identity/v3/projects/b4ba4c985f3344e3b0f640a1ec72b477 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 18 05:29:42.594311 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c61866-3e76-47cb-9f79-3e15cb836bbd None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:29:42.594833 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c61866-3e76-47cb-9f79-3e15cb836bbd None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:29:42.594947 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c61866-3e76-47cb-9f79-3e15cb836bbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:29:42.612938 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-c0c61866-3e76-47cb-9f79-3e15cb836bbd None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:29:42.614204 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-c0c61866-3e76-47cb-9f79-3e15cb836bbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:29:42.669012 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:29:42.675489 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1404/2830] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:29:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:29:42.685193 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7663eb5-b1ec-4474-9065-609406ef90b6 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:29:42.685713 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7663eb5-b1ec-4474-9065-609406ef90b6 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:29:42.685923 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7663eb5-b1ec-4474-9065-609406ef90b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:29:42.705068 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-c7663eb5-b1ec-4474-9065-609406ef90b6 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:29:42.706438 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-c7663eb5-b1ec-4474-9065-609406ef90b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:29:42.714662 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1427/2831] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:29:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:29:42.718824 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c61866-3e76-47cb-9f79-3e15cb836bbd None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:29:42.728522 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c61866-3e76-47cb-9f79-3e15cb836bbd None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:29:42.733320 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0eb069-bfe1-47f0-bfec-6e9e0e0fdbd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:29:42.734018 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0eb069-bfe1-47f0-bfec-6e9e0e0fdbd7 admin admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:29:42.734290 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0eb069-bfe1-47f0-bfec-6e9e0e0fdbd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:29:42.734566 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0eb069-bfe1-47f0-bfec-6e9e0e0fdbd7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:29:42.735314 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0eb069-bfe1-47f0-bfec-6e9e0e0fdbd7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 05:29:42.735629 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0eb069-bfe1-47f0-bfec-6e9e0e0fdbd7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:29:42.737249 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0eb069-bfe1-47f0-bfec-6e9e0e0fdbd7 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:29:42.749504 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1405/2832] 10.222.0.23 () {58 vars in 1152 bytes} [Sat Jun 18 05:29:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 05:29:42.753710 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7663eb5-b1ec-4474-9065-609406ef90b6 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:29:42.764423 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7663eb5-b1ec-4474-9065-609406ef90b6 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:29:42.770348 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62da487-aa00-4f4e-b79e-8be880124f02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:29:42.770348 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62da487-aa00-4f4e-b79e-8be880124f02 admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:29:42.770348 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62da487-aa00-4f4e-b79e-8be880124f02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:29:42.770348 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62da487-aa00-4f4e-b79e-8be880124f02 admin admin] PATH_INFO: `/v3/projects` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:29:42.770872 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62da487-aa00-4f4e-b79e-8be880124f02 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:29:42.772762 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62da487-aa00-4f4e-b79e-8be880124f02 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:29:42.896846 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1428/2833] 10.222.0.23 () {60 vars in 1152 bytes} [Sat Jun 18 05:29:42 2022] POST /identity/v3/projects => generated 379 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 18 05:29:42.900941 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0eb069-bfe1-47f0-bfec-6e9e0e0fdbd7 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:29:42.911376 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0eb069-bfe1-47f0-bfec-6e9e0e0fdbd7 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:29:42.967960 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791dd592-f59a-4a5f-9b18-e63d9bc7b0a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:29:42.968645 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791dd592-f59a-4a5f-9b18-e63d9bc7b0a6 admin admin] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:29:42.968921 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791dd592-f59a-4a5f-9b18-e63d9bc7b0a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:29:42.969195 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791dd592-f59a-4a5f-9b18-e63d9bc7b0a6 admin admin] PATH_INFO: `/v3/users` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:29:42.969908 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791dd592-f59a-4a5f-9b18-e63d9bc7b0a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:29:42.971873 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791dd592-f59a-4a5f-9b18-e63d9bc7b0a6 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:29:43.035528 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1406/2834] 10.222.0.23 () {60 vars in 1143 bytes} [Sat Jun 18 05:29:42 2022] POST /identity/v3/users => generated 290 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:29:43.039742 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62da487-aa00-4f4e-b79e-8be880124f02 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:29:43.050883 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62da487-aa00-4f4e-b79e-8be880124f02 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:29:43.055610 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5bdf4a-e4f2-4788-a550-b2748ee9f258 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:29:43.058999 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5bdf4a-e4f2-4788-a550-b2748ee9f258 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:29:43.058999 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5bdf4a-e4f2-4788-a550-b2748ee9f258 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:29:43.058999 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5bdf4a-e4f2-4788-a550-b2748ee9f258 admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:29:43.058999 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5bdf4a-e4f2-4788-a550-b2748ee9f258 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:29:43.059877 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5bdf4a-e4f2-4788-a550-b2748ee9f258 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:29:43.070177 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1429/2835] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:29:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:29:43.074357 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791dd592-f59a-4a5f-9b18-e63d9bc7b0a6 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:29:43.084312 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791dd592-f59a-4a5f-9b18-e63d9bc7b0a6 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:29:43.089004 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac09479-70e6-43c5-bf28-4bd40dbdfc8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:29:43.089704 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac09479-70e6-43c5-bf28-4bd40dbdfc8b admin admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:29:43.089990 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac09479-70e6-43c5-bf28-4bd40dbdfc8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:29:43.090274 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac09479-70e6-43c5-bf28-4bd40dbdfc8b admin admin] PATH_INFO: `/v3/roles` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:29:43.091000 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac09479-70e6-43c5-bf28-4bd40dbdfc8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:29:43.092591 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac09479-70e6-43c5-bf28-4bd40dbdfc8b admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:29:43.103525 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1407/2836] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:29:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:29:43.108237 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5bdf4a-e4f2-4788-a550-b2748ee9f258 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:29:43.118278 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5bdf4a-e4f2-4788-a550-b2748ee9f258 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:29:43.123791 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a02fcf4-c66e-46ef-aaf3-e59507b28a34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:29:43.124653 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a02fcf4-c66e-46ef-aaf3-e59507b28a34 admin admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:29:43.124936 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a02fcf4-c66e-46ef-aaf3-e59507b28a34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:29:43.125206 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a02fcf4-c66e-46ef-aaf3-e59507b28a34 admin admin] PATH_INFO: `/v3/projects/398c90b02fb54699b5563270c98b8234/users/f6207aa8cbcd48989b98204d53e59107/roles/cbf5c4c26898413d93e73b46b41046a3` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:29:43.141432 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a02fcf4-c66e-46ef-aaf3-e59507b28a34 admin admin] RBAC: Authorizing `identity:create_grant(project_id=398c90b02fb54699b5563270c98b8234, user_id=f6207aa8cbcd48989b98204d53e59107, role_id=cbf5c4c26898413d93e73b46b41046a3)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:29:43.144075 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a02fcf4-c66e-46ef-aaf3-e59507b28a34 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:29:43.215550 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1430/2837] 10.222.0.23 () {60 vars in 1482 bytes} [Sat Jun 18 05:29:43 2022] PUT /identity/v3/projects/398c90b02fb54699b5563270c98b8234/users/f6207aa8cbcd48989b98204d53e59107/roles/cbf5c4c26898413d93e73b46b41046a3 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 18 05:29:43.225911 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130ff874-f9e5-4e14-b76e-aba81d9da132 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:29:43.226203 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130ff874-f9e5-4e14-b76e-aba81d9da132 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:29:43.226486 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130ff874-f9e5-4e14-b76e-aba81d9da132 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:29:43.253950 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-130ff874-f9e5-4e14-b76e-aba81d9da132 None None] MFA Rules not processed for user `f6207aa8cbcd48989b98204d53e59107`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:29:43.255243 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-130ff874-f9e5-4e14-b76e-aba81d9da132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:29:43.318862 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1408/2838] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:29:43 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:29:43.323536 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a02fcf4-c66e-46ef-aaf3-e59507b28a34 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:29:43.348497 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:29:43.385968 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a02fcf4-c66e-46ef-aaf3-e59507b28a34 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:29:43.390716 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7820dd42-b22d-444c-bc05-b988ac19eb47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:29:43.391377 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7820dd42-b22d-444c-bc05-b988ac19eb47 admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:29:43.391659 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7820dd42-b22d-444c-bc05-b988ac19eb47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:29:43.392350 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7820dd42-b22d-444c-bc05-b988ac19eb47 admin admin] PATH_INFO: `/v3/projects` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:29:43.392605 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7820dd42-b22d-444c-bc05-b988ac19eb47 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:29:43.394575 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7820dd42-b22d-444c-bc05-b988ac19eb47 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:29:43.521412 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1431/2839] 10.222.0.23 () {60 vars in 1152 bytes} [Sat Jun 18 05:29:43 2022] POST /identity/v3/projects => generated 377 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:29:43.524895 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130ff874-f9e5-4e14-b76e-aba81d9da132 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:29:43.537484 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130ff874-f9e5-4e14-b76e-aba81d9da132 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:29:43.583831 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:29:43.593202 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814bac5d-2a1d-4ce0-8b12-b90891c4c6b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:29:43.593896 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814bac5d-2a1d-4ce0-8b12-b90891c4c6b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:29:43.594179 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814bac5d-2a1d-4ce0-8b12-b90891c4c6b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:29:43.594455 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814bac5d-2a1d-4ce0-8b12-b90891c4c6b8 admin admin] PATH_INFO: `/v3/users` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:29:43.595135 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814bac5d-2a1d-4ce0-8b12-b90891c4c6b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:29:43.597039 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814bac5d-2a1d-4ce0-8b12-b90891c4c6b8 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:29:43.731063 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1409/2840] 10.222.0.23 () {60 vars in 1143 bytes} [Sat Jun 18 05:29:43 2022] POST /identity/v3/users => generated 289 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:29:43.735022 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7820dd42-b22d-444c-bc05-b988ac19eb47 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:29:43.745532 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7820dd42-b22d-444c-bc05-b988ac19eb47 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:29:43.750451 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20eeadba-ecc2-4232-b950-fdf6bc344c95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:29:43.751096 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eeadba-ecc2-4232-b950-fdf6bc344c95 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:29:43.756446 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eeadba-ecc2-4232-b950-fdf6bc344c95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:29:43.756446 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eeadba-ecc2-4232-b950-fdf6bc344c95 admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:29:43.756446 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20eeadba-ecc2-4232-b950-fdf6bc344c95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:29:43.756446 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20eeadba-ecc2-4232-b950-fdf6bc344c95 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:29:43.765429 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1432/2841] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:29:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:29:43.769302 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814bac5d-2a1d-4ce0-8b12-b90891c4c6b8 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:29:43.780024 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814bac5d-2a1d-4ce0-8b12-b90891c4c6b8 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:29:43.784790 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc30f82-cf2f-49ad-a5e3-ff6244796e1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:29:43.785468 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc30f82-cf2f-49ad-a5e3-ff6244796e1a admin admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:29:43.785758 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc30f82-cf2f-49ad-a5e3-ff6244796e1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:29:43.786036 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc30f82-cf2f-49ad-a5e3-ff6244796e1a admin admin] PATH_INFO: `/v3/roles` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:29:43.786796 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc30f82-cf2f-49ad-a5e3-ff6244796e1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:29:43.788389 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc30f82-cf2f-49ad-a5e3-ff6244796e1a admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:29:43.798125 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1410/2842] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:29:43 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 18 05:29:43.802257 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20eeadba-ecc2-4232-b950-fdf6bc344c95 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:29:43.812029 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20eeadba-ecc2-4232-b950-fdf6bc344c95 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:29:43.816768 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710105b9-4fd5-4952-b8de-9b10aaec6e3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:29:43.817667 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710105b9-4fd5-4952-b8de-9b10aaec6e3b admin admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:29:43.817954 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710105b9-4fd5-4952-b8de-9b10aaec6e3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:29:43.818229 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710105b9-4fd5-4952-b8de-9b10aaec6e3b admin admin] PATH_INFO: `/v3/projects/6d62559d75704c12ac666b7292ee1347/users/1b4c7245c3874123ac89a3cad9764889/roles/cbf5c4c26898413d93e73b46b41046a3` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:29:43.834157 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710105b9-4fd5-4952-b8de-9b10aaec6e3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d62559d75704c12ac666b7292ee1347, user_id=1b4c7245c3874123ac89a3cad9764889, role_id=cbf5c4c26898413d93e73b46b41046a3)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:29:43.836798 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710105b9-4fd5-4952-b8de-9b10aaec6e3b admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:29:43.910016 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1433/2843] 10.222.0.23 () {60 vars in 1482 bytes} [Sat Jun 18 05:29:43 2022] PUT /identity/v3/projects/6d62559d75704c12ac666b7292ee1347/users/1b4c7245c3874123ac89a3cad9764889/roles/cbf5c4c26898413d93e73b46b41046a3 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:29:43.920220 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc61867-dc21-4e45-bf7b-888aa6b9efb8 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:29:43.920509 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc61867-dc21-4e45-bf7b-888aa6b9efb8 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:29:43.920816 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc61867-dc21-4e45-bf7b-888aa6b9efb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:29:43.946350 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-9fc61867-dc21-4e45-bf7b-888aa6b9efb8 None None] MFA Rules not processed for user `1b4c7245c3874123ac89a3cad9764889`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:29:43.948048 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-9fc61867-dc21-4e45-bf7b-888aa6b9efb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:29:44.011166 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1411/2844] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:29:43 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:29:44.015637 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710105b9-4fd5-4952-b8de-9b10aaec6e3b admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:29:44.075566 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710105b9-4fd5-4952-b8de-9b10aaec6e3b admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:29:44.080154 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f48017-d59a-4a49-ab77-41b83a48f3ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:29:44.080797 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f48017-d59a-4a49-ab77-41b83a48f3ed admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:29:44.081074 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f48017-d59a-4a49-ab77-41b83a48f3ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:29:44.081346 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f48017-d59a-4a49-ab77-41b83a48f3ed admin admin] PATH_INFO: `/v3/projects` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:29:44.082055 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f48017-d59a-4a49-ab77-41b83a48f3ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:29:44.083940 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f48017-d59a-4a49-ab77-41b83a48f3ed admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:29:44.183158 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1434/2845] 10.222.0.23 () {60 vars in 1152 bytes} [Sat Jun 18 05:29:44 2022] POST /identity/v3/projects => generated 379 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:29:44.187380 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc61867-dc21-4e45-bf7b-888aa6b9efb8 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:29:44.198757 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc61867-dc21-4e45-bf7b-888aa6b9efb8 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:29:44.255112 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3fd0ac-9b3d-4fc2-b11e-186519f1e7af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:29:44.255764 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3fd0ac-9b3d-4fc2-b11e-186519f1e7af admin admin] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:29:44.256045 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3fd0ac-9b3d-4fc2-b11e-186519f1e7af admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:29:44.256318 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3fd0ac-9b3d-4fc2-b11e-186519f1e7af admin admin] PATH_INFO: `/v3/users` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:29:44.256993 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3fd0ac-9b3d-4fc2-b11e-186519f1e7af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:29:44.258921 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3fd0ac-9b3d-4fc2-b11e-186519f1e7af admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:29:44.344362 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1412/2846] 10.222.0.23 () {60 vars in 1143 bytes} [Sat Jun 18 05:29:44 2022] POST /identity/v3/users => generated 296 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:29:44.348290 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f48017-d59a-4a49-ab77-41b83a48f3ed admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:29:44.358645 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f48017-d59a-4a49-ab77-41b83a48f3ed admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:29:44.363356 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c78bc39-6ef7-4793-9a64-145ff1e39303 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:29:44.364011 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c78bc39-6ef7-4793-9a64-145ff1e39303 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:29:44.364293 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c78bc39-6ef7-4793-9a64-145ff1e39303 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:29:44.364570 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c78bc39-6ef7-4793-9a64-145ff1e39303 admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:29:44.365286 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c78bc39-6ef7-4793-9a64-145ff1e39303 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:29:44.366933 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c78bc39-6ef7-4793-9a64-145ff1e39303 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:29:44.378082 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1435/2847] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:29:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:29:44.382162 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3fd0ac-9b3d-4fc2-b11e-186519f1e7af admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:29:44.392959 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3fd0ac-9b3d-4fc2-b11e-186519f1e7af admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:29:44.398542 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938f358e-a961-4e5f-b8b1-782c1e64fc32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:29:44.399408 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938f358e-a961-4e5f-b8b1-782c1e64fc32 admin admin] REQUEST_METHOD: `PUT` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:29:44.399690 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938f358e-a961-4e5f-b8b1-782c1e64fc32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:29:44.399963 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938f358e-a961-4e5f-b8b1-782c1e64fc32 admin admin] PATH_INFO: `/v3/projects/0c24fb2a133241668620055926c9da83/users/f914a8b864904987a7d22ffefe8efab6/roles/21c65f2c51bd4dd283d13507d2714467` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:29:44.415001 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938f358e-a961-4e5f-b8b1-782c1e64fc32 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c24fb2a133241668620055926c9da83, user_id=f914a8b864904987a7d22ffefe8efab6, role_id=21c65f2c51bd4dd283d13507d2714467)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:29:44.417850 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938f358e-a961-4e5f-b8b1-782c1e64fc32 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:29:44.482139 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1413/2848] 10.222.0.23 () {60 vars in 1482 bytes} [Sat Jun 18 05:29:44 2022] PUT /identity/v3/projects/0c24fb2a133241668620055926c9da83/users/f914a8b864904987a7d22ffefe8efab6/roles/21c65f2c51bd4dd283d13507d2714467 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:29:44.486035 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c78bc39-6ef7-4793-9a64-145ff1e39303 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:29:44.496948 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c78bc39-6ef7-4793-9a64-145ff1e39303 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:29:44.540233 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:29:44.553552 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a727e836-389e-44a4-8401-ea56830ce5e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:29:44.554210 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a727e836-389e-44a4-8401-ea56830ce5e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:29:44.554491 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a727e836-389e-44a4-8401-ea56830ce5e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:29:44.554761 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a727e836-389e-44a4-8401-ea56830ce5e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:29:44.555474 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a727e836-389e-44a4-8401-ea56830ce5e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:29:44.558155 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a727e836-389e-44a4-8401-ea56830ce5e1 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:29:44.567856 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1436/2849] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:29:44 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:29:44.572012 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938f358e-a961-4e5f-b8b1-782c1e64fc32 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:29:44.582531 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938f358e-a961-4e5f-b8b1-782c1e64fc32 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:29:44.587175 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86037d5-1907-489e-8b1d-5401910dea9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:29:44.588047 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86037d5-1907-489e-8b1d-5401910dea9d admin admin] REQUEST_METHOD: `PUT` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:29:44.588394 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86037d5-1907-489e-8b1d-5401910dea9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:29:44.588871 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86037d5-1907-489e-8b1d-5401910dea9d admin admin] PATH_INFO: `/v3/projects/0c24fb2a133241668620055926c9da83/users/f914a8b864904987a7d22ffefe8efab6/roles/21c65f2c51bd4dd283d13507d2714467` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:29:44.590779 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86037d5-1907-489e-8b1d-5401910dea9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c24fb2a133241668620055926c9da83, user_id=f914a8b864904987a7d22ffefe8efab6, role_id=21c65f2c51bd4dd283d13507d2714467)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:29:44.593455 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86037d5-1907-489e-8b1d-5401910dea9d admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:29:44.604795 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1414/2850] 10.222.0.23 () {60 vars in 1482 bytes} [Sat Jun 18 05:29:44 2022] PUT /identity/v3/projects/0c24fb2a133241668620055926c9da83/users/f914a8b864904987a7d22ffefe8efab6/roles/21c65f2c51bd4dd283d13507d2714467 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:29:44.665713 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d5c631-8ef3-4630-bae6-f5bb81f231df None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:29:44.666014 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d5c631-8ef3-4630-bae6-f5bb81f231df None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:29:44.666783 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d5c631-8ef3-4630-bae6-f5bb81f231df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:29:44.693760 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-29d5c631-8ef3-4630-bae6-f5bb81f231df None None] MFA Rules not processed for user `f914a8b864904987a7d22ffefe8efab6`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:29:44.694963 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-29d5c631-8ef3-4630-bae6-f5bb81f231df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:29:44.761740 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1437/2851] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:29:44 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:29:44.772820 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86037d5-1907-489e-8b1d-5401910dea9d admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:29:44.774541 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-a86037d5-1907-489e-8b1d-5401910dea9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:29:44.788977 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:29:44.832428 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86037d5-1907-489e-8b1d-5401910dea9d admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:29:44.836090 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ea34a4-632c-4121-b94e-565344a564f1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9de0428abc8400aa338aba4f7c34c94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:29:44.836719 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ea34a4-632c-4121-b94e-565344a564f1 service cinder] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:29:44.837002 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ea34a4-632c-4121-b94e-565344a564f1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:29:44.837275 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ea34a4-632c-4121-b94e-565344a564f1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:29:44.846759 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ea34a4-632c-4121-b94e-565344a564f1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:29:44.848183 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ea34a4-632c-4121-b94e-565344a564f1 service cinder] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:29:44.858745 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1415/2852] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:29:44 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:29:44.974491 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d5c631-8ef3-4630-bae6-f5bb81f231df None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:29:44.975912 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-29d5c631-8ef3-4630-bae6-f5bb81f231df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:29:44.988614 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d5c631-8ef3-4630-bae6-f5bb81f231df None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:29:44.992368 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129cae5d-c39a-4017-a95d-91c51465347a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9de0428abc8400aa338aba4f7c34c94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:29:44.992998 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129cae5d-c39a-4017-a95d-91c51465347a service cinder] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:29:44.993280 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129cae5d-c39a-4017-a95d-91c51465347a service cinder] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:29:44.993582 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129cae5d-c39a-4017-a95d-91c51465347a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:29:45.050001 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129cae5d-c39a-4017-a95d-91c51465347a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:29:45.051474 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129cae5d-c39a-4017-a95d-91c51465347a service cinder] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:29:45.060682 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1438/2853] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:29:44 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:29:49.115965 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ea34a4-632c-4121-b94e-565344a564f1 service cinder] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:29:49.182555 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ea34a4-632c-4121-b94e-565344a564f1 service cinder] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:29:49.187254 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff08b09-d5bd-4d56-96bc-374a5ab44128 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:29:49.188047 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff08b09-d5bd-4d56-96bc-374a5ab44128 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:29:49.188330 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff08b09-d5bd-4d56-96bc-374a5ab44128 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:29:49.188608 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff08b09-d5bd-4d56-96bc-374a5ab44128 admin admin] PATH_INFO: `/v3/users/f6207aa8cbcd48989b98204d53e59107` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:29:49.189828 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff08b09-d5bd-4d56-96bc-374a5ab44128 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6207aa8cbcd48989b98204d53e59107)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:29:49.191750 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff08b09-d5bd-4d56-96bc-374a5ab44128 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:29:49.329077 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-eff08b09-d5bd-4d56-96bc-374a5ab44128 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6207aa8cbcd48989b98204d53e59107'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:29:49.329750 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-eff08b09-d5bd-4d56-96bc-374a5ab44128 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6207aa8cbcd48989b98204d53e59107'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:29:49.372380 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-eff08b09-d5bd-4d56-96bc-374a5ab44128 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6207aa8cbcd48989b98204d53e59107'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:29:49.385279 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-eff08b09-d5bd-4d56-96bc-374a5ab44128 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6207aa8cbcd48989b98204d53e59107'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:29:49.393985 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-eff08b09-d5bd-4d56-96bc-374a5ab44128 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6207aa8cbcd48989b98204d53e59107'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:29:49.399513 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1416/2854] 10.222.0.23 () {58 vars in 1223 bytes} [Sat Jun 18 05:29:49 2022] DELETE /identity/v3/users/f6207aa8cbcd48989b98204d53e59107 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:29:49.404039 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70e67de-fbfd-47be-a2a1-45734edfc471 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:29:49.404476 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70e67de-fbfd-47be-a2a1-45734edfc471 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:29:49.404590 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70e67de-fbfd-47be-a2a1-45734edfc471 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:29:49.422065 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-a70e67de-fbfd-47be-a2a1-45734edfc471 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:29:49.423312 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-a70e67de-fbfd-47be-a2a1-45734edfc471 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:29:49.482168 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1439/2855] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:29:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:29:49.491787 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff08b09-d5bd-4d56-96bc-374a5ab44128 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:29:49.493274 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-eff08b09-d5bd-4d56-96bc-374a5ab44128 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:29:49.533602 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:29:49.550923 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff08b09-d5bd-4d56-96bc-374a5ab44128 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:29:49.554625 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac564df-bc5b-45fa-a727-8d1fd31e36e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c4d7cde8f4ad9aa479bb09e325db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:29:49.555258 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac564df-bc5b-45fa-a727-8d1fd31e36e1 service neutron] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:29:49.555540 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac564df-bc5b-45fa-a727-8d1fd31e36e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:29:49.555813 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac564df-bc5b-45fa-a727-8d1fd31e36e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:29:49.565982 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac564df-bc5b-45fa-a727-8d1fd31e36e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:29:49.567399 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac564df-bc5b-45fa-a727-8d1fd31e36e1 service neutron] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:29:49.576666 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1417/2856] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:29:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:29:50.001093 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70e67de-fbfd-47be-a2a1-45734edfc471 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:29:50.002675 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-a70e67de-fbfd-47be-a2a1-45734edfc471 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:29:50.019299 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70e67de-fbfd-47be-a2a1-45734edfc471 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:29:50.023093 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56c816e-133e-4519-8ec4-d3d5708539e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:29:50.023952 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56c816e-133e-4519-8ec4-d3d5708539e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:29:50.024264 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56c816e-133e-4519-8ec4-d3d5708539e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:29:50.024558 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56c816e-133e-4519-8ec4-d3d5708539e8 admin admin] PATH_INFO: `/v3/projects/398c90b02fb54699b5563270c98b8234` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:29:50.025849 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56c816e-133e-4519-8ec4-d3d5708539e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=398c90b02fb54699b5563270c98b8234)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:29:50.027756 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56c816e-133e-4519-8ec4-d3d5708539e8 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:29:50.029815 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:29:50.129614 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-b56c816e-133e-4519-8ec4-d3d5708539e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '398c90b02fb54699b5563270c98b8234', 'request_id': 'req-b56c816e-133e-4519-8ec4-d3d5708539e8'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:29:50.135284 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-b56c816e-133e-4519-8ec4-d3d5708539e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '398c90b02fb54699b5563270c98b8234', 'request_id': 'req-b56c816e-133e-4519-8ec4-d3d5708539e8'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:29:50.141651 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-b56c816e-133e-4519-8ec4-d3d5708539e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '398c90b02fb54699b5563270c98b8234', 'request_id': 'req-b56c816e-133e-4519-8ec4-d3d5708539e8'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:29:50.232484 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-b56c816e-133e-4519-8ec4-d3d5708539e8 admin admin] The token cache is being invalidate because project 398c90b02fb54699b5563270c98b8234 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80469) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 18 05:29:50.232775 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-b56c816e-133e-4519-8ec4-d3d5708539e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:29:50.234015 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-b56c816e-133e-4519-8ec4-d3d5708539e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:29:50.235584 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1440/2857] 10.222.0.23 () {58 vars in 1232 bytes} [Sat Jun 18 05:29:49 2022] DELETE /identity/v3/projects/398c90b02fb54699b5563270c98b8234 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:29:50.240006 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac564df-bc5b-45fa-a727-8d1fd31e36e1 service neutron] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:29:50.250563 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac564df-bc5b-45fa-a727-8d1fd31e36e1 service neutron] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:29:50.310255 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b303fca-0357-4ff2-a9cd-f65e806f2a19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:29:50.311117 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b303fca-0357-4ff2-a9cd-f65e806f2a19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:29:50.311620 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b303fca-0357-4ff2-a9cd-f65e806f2a19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:29:50.312205 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b303fca-0357-4ff2-a9cd-f65e806f2a19 admin admin] PATH_INFO: `/v3/users/1b4c7245c3874123ac89a3cad9764889` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:29:50.313514 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b303fca-0357-4ff2-a9cd-f65e806f2a19 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b4c7245c3874123ac89a3cad9764889)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:29:50.315313 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b303fca-0357-4ff2-a9cd-f65e806f2a19 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:29:50.449182 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-7b303fca-0357-4ff2-a9cd-f65e806f2a19 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b4c7245c3874123ac89a3cad9764889'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:29:50.449875 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-7b303fca-0357-4ff2-a9cd-f65e806f2a19 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b4c7245c3874123ac89a3cad9764889'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:29:50.515873 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-7b303fca-0357-4ff2-a9cd-f65e806f2a19 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b4c7245c3874123ac89a3cad9764889'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:29:50.527913 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-7b303fca-0357-4ff2-a9cd-f65e806f2a19 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b4c7245c3874123ac89a3cad9764889'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:29:50.535615 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-7b303fca-0357-4ff2-a9cd-f65e806f2a19 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b4c7245c3874123ac89a3cad9764889'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:29:50.541634 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1418/2858] 10.222.0.23 () {58 vars in 1223 bytes} [Sat Jun 18 05:29:50 2022] DELETE /identity/v3/users/1b4c7245c3874123ac89a3cad9764889 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 18 05:29:50.912035 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56c816e-133e-4519-8ec4-d3d5708539e8 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:29:50.915179 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-b56c816e-133e-4519-8ec4-d3d5708539e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:29:50.929461 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56c816e-133e-4519-8ec4-d3d5708539e8 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:29:50.988145 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1af663-e23a-414c-b4e2-b942fc7e3b15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:29:50.993502 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1af663-e23a-414c-b4e2-b942fc7e3b15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:29:50.993502 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1af663-e23a-414c-b4e2-b942fc7e3b15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:29:50.993502 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1af663-e23a-414c-b4e2-b942fc7e3b15 admin admin] PATH_INFO: `/v3/projects/6d62559d75704c12ac666b7292ee1347` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:29:50.993502 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1af663-e23a-414c-b4e2-b942fc7e3b15 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d62559d75704c12ac666b7292ee1347)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:29:50.993502 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1af663-e23a-414c-b4e2-b942fc7e3b15 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:29:51.109782 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-6b1af663-e23a-414c-b4e2-b942fc7e3b15 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d62559d75704c12ac666b7292ee1347', 'request_id': 'req-6b1af663-e23a-414c-b4e2-b942fc7e3b15'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:29:51.115298 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-6b1af663-e23a-414c-b4e2-b942fc7e3b15 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d62559d75704c12ac666b7292ee1347', 'request_id': 'req-6b1af663-e23a-414c-b4e2-b942fc7e3b15'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:29:51.121460 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-6b1af663-e23a-414c-b4e2-b942fc7e3b15 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d62559d75704c12ac666b7292ee1347', 'request_id': 'req-6b1af663-e23a-414c-b4e2-b942fc7e3b15'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:29:51.229553 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-6b1af663-e23a-414c-b4e2-b942fc7e3b15 admin admin] The token cache is being invalidate because project 6d62559d75704c12ac666b7292ee1347 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80469) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 18 05:29:51.229842 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-6b1af663-e23a-414c-b4e2-b942fc7e3b15 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:29:51.230477 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-6b1af663-e23a-414c-b4e2-b942fc7e3b15 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:29:51.232031 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1441/2859] 10.222.0.23 () {58 vars in 1232 bytes} [Sat Jun 18 05:29:50 2022] DELETE /identity/v3/projects/6d62559d75704c12ac666b7292ee1347 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:29:51.237129 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b303fca-0357-4ff2-a9cd-f65e806f2a19 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:29:51.238540 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-7b303fca-0357-4ff2-a9cd-f65e806f2a19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:29:51.252424 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b303fca-0357-4ff2-a9cd-f65e806f2a19 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:29:51.303706 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:29:51.309400 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca128668-dcc1-42ad-bd93-6efbd4d9be3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:29:51.310271 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca128668-dcc1-42ad-bd93-6efbd4d9be3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:29:51.310551 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca128668-dcc1-42ad-bd93-6efbd4d9be3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:29:51.310830 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca128668-dcc1-42ad-bd93-6efbd4d9be3b admin admin] PATH_INFO: `/v3/users/f914a8b864904987a7d22ffefe8efab6` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:29:51.313107 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca128668-dcc1-42ad-bd93-6efbd4d9be3b admin admin] RBAC: Authorizing `identity:delete_user(user_id=f914a8b864904987a7d22ffefe8efab6)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:29:51.315070 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca128668-dcc1-42ad-bd93-6efbd4d9be3b admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:29:51.541348 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-ca128668-dcc1-42ad-bd93-6efbd4d9be3b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f914a8b864904987a7d22ffefe8efab6'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:29:51.542036 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-ca128668-dcc1-42ad-bd93-6efbd4d9be3b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f914a8b864904987a7d22ffefe8efab6'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:29:51.644214 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-ca128668-dcc1-42ad-bd93-6efbd4d9be3b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f914a8b864904987a7d22ffefe8efab6'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:29:51.657932 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-ca128668-dcc1-42ad-bd93-6efbd4d9be3b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f914a8b864904987a7d22ffefe8efab6'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:29:51.664979 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-ca128668-dcc1-42ad-bd93-6efbd4d9be3b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f914a8b864904987a7d22ffefe8efab6'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:29:51.671730 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1419/2860] 10.222.0.23 () {58 vars in 1223 bytes} [Sat Jun 18 05:29:51 2022] DELETE /identity/v3/users/f914a8b864904987a7d22ffefe8efab6 => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:29:52.085989 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1af663-e23a-414c-b4e2-b942fc7e3b15 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:29:52.085989 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-6b1af663-e23a-414c-b4e2-b942fc7e3b15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:29:52.096857 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1af663-e23a-414c-b4e2-b942fc7e3b15 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:29:52.115226 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:29:52.155631 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef15551-3d9e-410a-8d0a-95f19c5127b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:29:52.156479 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef15551-3d9e-410a-8d0a-95f19c5127b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:29:52.156772 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef15551-3d9e-410a-8d0a-95f19c5127b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:29:52.157049 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef15551-3d9e-410a-8d0a-95f19c5127b8 admin admin] PATH_INFO: `/v3/projects/0c24fb2a133241668620055926c9da83` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:29:52.158298 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef15551-3d9e-410a-8d0a-95f19c5127b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c24fb2a133241668620055926c9da83)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:29:52.160509 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef15551-3d9e-410a-8d0a-95f19c5127b8 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:29:52.273327 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-8ef15551-3d9e-410a-8d0a-95f19c5127b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c24fb2a133241668620055926c9da83', 'request_id': 'req-8ef15551-3d9e-410a-8d0a-95f19c5127b8'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:29:52.278912 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-8ef15551-3d9e-410a-8d0a-95f19c5127b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c24fb2a133241668620055926c9da83', 'request_id': 'req-8ef15551-3d9e-410a-8d0a-95f19c5127b8'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:29:52.285613 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-8ef15551-3d9e-410a-8d0a-95f19c5127b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c24fb2a133241668620055926c9da83', 'request_id': 'req-8ef15551-3d9e-410a-8d0a-95f19c5127b8'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:29:52.392962 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-8ef15551-3d9e-410a-8d0a-95f19c5127b8 admin admin] The token cache is being invalidate because project 0c24fb2a133241668620055926c9da83 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80469) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 18 05:29:52.392962 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-8ef15551-3d9e-410a-8d0a-95f19c5127b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:29:52.393638 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-8ef15551-3d9e-410a-8d0a-95f19c5127b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:29:52.399836 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1442/2861] 10.222.0.23 () {58 vars in 1232 bytes} [Sat Jun 18 05:29:52 2022] DELETE /identity/v3/projects/0c24fb2a133241668620055926c9da83 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:29:52.401172 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded514a5-1ba2-45c4-b903-1c57df992a58 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:29:52.401603 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded514a5-1ba2-45c4-b903-1c57df992a58 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:29:52.401776 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded514a5-1ba2-45c4-b903-1c57df992a58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:29:52.419802 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-ded514a5-1ba2-45c4-b903-1c57df992a58 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:29:52.421022 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-ded514a5-1ba2-45c4-b903-1c57df992a58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:29:52.480690 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1420/2862] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:29:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:29:52.488927 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf512dbe-cb3b-460a-a428-ea7bffb3d97e None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:29:52.489215 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf512dbe-cb3b-460a-a428-ea7bffb3d97e None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:29:52.489527 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf512dbe-cb3b-460a-a428-ea7bffb3d97e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:29:52.507989 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-bf512dbe-cb3b-460a-a428-ea7bffb3d97e None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:29:52.509192 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-bf512dbe-cb3b-460a-a428-ea7bffb3d97e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:29:52.517237 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1443/2863] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:29:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:29:52.521423 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded514a5-1ba2-45c4-b903-1c57df992a58 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:29:52.531831 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded514a5-1ba2-45c4-b903-1c57df992a58 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:29:52.536634 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a57d22-83a1-48f5-915a-549380a3abb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:29:52.537279 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a57d22-83a1-48f5-915a-549380a3abb4 admin admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:29:52.537605 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a57d22-83a1-48f5-915a-549380a3abb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:29:52.537887 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a57d22-83a1-48f5-915a-549380a3abb4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:29:52.538612 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a57d22-83a1-48f5-915a-549380a3abb4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 05:29:52.538925 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a57d22-83a1-48f5-915a-549380a3abb4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:29:52.540525 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a57d22-83a1-48f5-915a-549380a3abb4 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:29:52.552512 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1421/2864] 10.222.0.23 () {58 vars in 1152 bytes} [Sat Jun 18 05:29:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 05:29:52.556704 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf512dbe-cb3b-460a-a428-ea7bffb3d97e None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:29:52.566084 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf512dbe-cb3b-460a-a428-ea7bffb3d97e None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:29:52.570936 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51a0173-4f67-42c6-9be3-45fdad840d19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:29:52.571587 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51a0173-4f67-42c6-9be3-45fdad840d19 admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:29:52.571871 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51a0173-4f67-42c6-9be3-45fdad840d19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:29:52.572146 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51a0173-4f67-42c6-9be3-45fdad840d19 admin admin] PATH_INFO: `/v3/projects` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:29:52.572835 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51a0173-4f67-42c6-9be3-45fdad840d19 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:29:52.574762 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51a0173-4f67-42c6-9be3-45fdad840d19 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:29:52.685913 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1444/2865] 10.222.0.23 () {60 vars in 1152 bytes} [Sat Jun 18 05:29:52 2022] POST /identity/v3/projects => generated 385 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:29:52.690125 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a57d22-83a1-48f5-915a-549380a3abb4 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:29:52.700043 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a57d22-83a1-48f5-915a-549380a3abb4 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:29:52.756867 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d934500-221d-4502-ab01-e0de54c96196 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:29:52.757564 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d934500-221d-4502-ab01-e0de54c96196 admin admin] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:29:52.757853 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d934500-221d-4502-ab01-e0de54c96196 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:29:52.758133 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d934500-221d-4502-ab01-e0de54c96196 admin admin] PATH_INFO: `/v3/users` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:29:52.758831 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d934500-221d-4502-ab01-e0de54c96196 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:29:52.760739 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d934500-221d-4502-ab01-e0de54c96196 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:29:52.769732 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:29:52.852612 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1422/2866] 10.222.0.23 () {60 vars in 1143 bytes} [Sat Jun 18 05:29:52 2022] POST /identity/v3/users => generated 293 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:29:52.856629 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51a0173-4f67-42c6-9be3-45fdad840d19 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:29:52.867228 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51a0173-4f67-42c6-9be3-45fdad840d19 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:29:52.872127 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f22072-e711-4324-b7ae-c8625d41a053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:29:52.872778 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f22072-e711-4324-b7ae-c8625d41a053 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:29:52.873058 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f22072-e711-4324-b7ae-c8625d41a053 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:29:52.874185 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f22072-e711-4324-b7ae-c8625d41a053 admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:29:52.874185 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f22072-e711-4324-b7ae-c8625d41a053 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:29:52.875732 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f22072-e711-4324-b7ae-c8625d41a053 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:29:52.885991 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1445/2867] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:29:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:29:52.890526 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d934500-221d-4502-ab01-e0de54c96196 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:29:52.900702 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d934500-221d-4502-ab01-e0de54c96196 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:29:52.906119 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbf3f74-0eaf-4aee-ad8c-17103e9932b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:29:52.906782 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbf3f74-0eaf-4aee-ad8c-17103e9932b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:29:52.907046 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbf3f74-0eaf-4aee-ad8c-17103e9932b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:29:52.907320 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbf3f74-0eaf-4aee-ad8c-17103e9932b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:29:52.908035 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbf3f74-0eaf-4aee-ad8c-17103e9932b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:29:52.909906 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbf3f74-0eaf-4aee-ad8c-17103e9932b8 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:29:52.920727 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1423/2868] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:29:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:29:52.934201 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f22072-e711-4324-b7ae-c8625d41a053 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:29:52.944416 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f22072-e711-4324-b7ae-c8625d41a053 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:29:52.949068 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b6bfd9-8991-4060-ae15-414645eedf94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:29:52.949960 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b6bfd9-8991-4060-ae15-414645eedf94 admin admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:29:52.950246 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b6bfd9-8991-4060-ae15-414645eedf94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:29:52.950520 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b6bfd9-8991-4060-ae15-414645eedf94 admin admin] PATH_INFO: `/v3/projects/3d4606866f934cfdb36bb9706914feb7/users/dae264d9829e4690894a764078ed7ebf/roles/cbf5c4c26898413d93e73b46b41046a3` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:29:52.965605 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b6bfd9-8991-4060-ae15-414645eedf94 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d4606866f934cfdb36bb9706914feb7, user_id=dae264d9829e4690894a764078ed7ebf, role_id=cbf5c4c26898413d93e73b46b41046a3)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:29:52.968934 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b6bfd9-8991-4060-ae15-414645eedf94 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:29:53.064814 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1446/2869] 10.222.0.23 () {60 vars in 1482 bytes} [Sat Jun 18 05:29:52 2022] PUT /identity/v3/projects/3d4606866f934cfdb36bb9706914feb7/users/dae264d9829e4690894a764078ed7ebf/roles/cbf5c4c26898413d93e73b46b41046a3 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:29:53.084992 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4cde6c-e61e-4388-ae73-e146141f7f51 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:29:53.085279 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4cde6c-e61e-4388-ae73-e146141f7f51 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:29:53.085591 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4cde6c-e61e-4388-ae73-e146141f7f51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:29:53.112899 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-ca4cde6c-e61e-4388-ae73-e146141f7f51 None None] MFA Rules not processed for user `dae264d9829e4690894a764078ed7ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:29:53.114061 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-ca4cde6c-e61e-4388-ae73-e146141f7f51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:29:53.180235 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1424/2870] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:29:53 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:29:53.184845 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b6bfd9-8991-4060-ae15-414645eedf94 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:29:53.232622 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:29:53.246461 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b6bfd9-8991-4060-ae15-414645eedf94 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:29:53.251860 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58d9ebe-a777-4031-9793-7684fc2c53e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:29:53.252511 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58d9ebe-a777-4031-9793-7684fc2c53e2 admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:29:53.252796 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58d9ebe-a777-4031-9793-7684fc2c53e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:29:53.253073 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58d9ebe-a777-4031-9793-7684fc2c53e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:29:53.253801 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58d9ebe-a777-4031-9793-7684fc2c53e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:29:53.255710 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58d9ebe-a777-4031-9793-7684fc2c53e2 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:29:53.359543 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1447/2871] 10.222.0.23 () {60 vars in 1152 bytes} [Sat Jun 18 05:29:53 2022] POST /identity/v3/projects => generated 383 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:29:53.364012 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4cde6c-e61e-4388-ae73-e146141f7f51 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:29:53.375816 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4cde6c-e61e-4388-ae73-e146141f7f51 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:29:53.434528 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f3b379-163e-4b6d-ac33-1dafc1737cd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:29:53.435197 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f3b379-163e-4b6d-ac33-1dafc1737cd1 admin admin] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:29:53.435483 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f3b379-163e-4b6d-ac33-1dafc1737cd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:29:53.435760 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f3b379-163e-4b6d-ac33-1dafc1737cd1 admin admin] PATH_INFO: `/v3/users` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:29:53.437038 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f3b379-163e-4b6d-ac33-1dafc1737cd1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:29:53.438995 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f3b379-163e-4b6d-ac33-1dafc1737cd1 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:29:53.512138 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1425/2872] 10.222.0.23 () {60 vars in 1143 bytes} [Sat Jun 18 05:29:53 2022] POST /identity/v3/users => generated 298 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:29:53.516287 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58d9ebe-a777-4031-9793-7684fc2c53e2 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:29:53.527154 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58d9ebe-a777-4031-9793-7684fc2c53e2 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:29:53.532962 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8344479a-7129-44e7-be4c-e2e69385641d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:29:53.533664 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8344479a-7129-44e7-be4c-e2e69385641d admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:29:53.533951 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8344479a-7129-44e7-be4c-e2e69385641d admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:29:53.534232 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8344479a-7129-44e7-be4c-e2e69385641d admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:29:53.535217 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8344479a-7129-44e7-be4c-e2e69385641d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:29:53.536823 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8344479a-7129-44e7-be4c-e2e69385641d admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:29:53.547917 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1448/2873] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:29:53 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:29:53.553669 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f3b379-163e-4b6d-ac33-1dafc1737cd1 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:29:53.563318 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f3b379-163e-4b6d-ac33-1dafc1737cd1 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:29:53.568185 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e07aa4f-91bb-4be8-b0e5-4cdf80cd0cbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:29:53.569056 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e07aa4f-91bb-4be8-b0e5-4cdf80cd0cbd admin admin] REQUEST_METHOD: `PUT` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:29:53.569346 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e07aa4f-91bb-4be8-b0e5-4cdf80cd0cbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:29:53.569651 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e07aa4f-91bb-4be8-b0e5-4cdf80cd0cbd admin admin] PATH_INFO: `/v3/projects/1c395e7385e1411ca7ae32e1b624d2e7/users/4ad59438688d42a9b5f64604ed88cc64/roles/21c65f2c51bd4dd283d13507d2714467` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:29:53.584584 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e07aa4f-91bb-4be8-b0e5-4cdf80cd0cbd admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c395e7385e1411ca7ae32e1b624d2e7, user_id=4ad59438688d42a9b5f64604ed88cc64, role_id=21c65f2c51bd4dd283d13507d2714467)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:29:53.587291 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e07aa4f-91bb-4be8-b0e5-4cdf80cd0cbd admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:29:53.649475 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1426/2874] 10.222.0.23 () {60 vars in 1482 bytes} [Sat Jun 18 05:29:53 2022] PUT /identity/v3/projects/1c395e7385e1411ca7ae32e1b624d2e7/users/4ad59438688d42a9b5f64604ed88cc64/roles/21c65f2c51bd4dd283d13507d2714467 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:29:53.653442 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8344479a-7129-44e7-be4c-e2e69385641d admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:29:53.663941 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8344479a-7129-44e7-be4c-e2e69385641d admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:29:53.719272 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3faf58-8784-40b3-ba16-12f67dbe7d99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:29:53.719927 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3faf58-8784-40b3-ba16-12f67dbe7d99 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:29:53.720210 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3faf58-8784-40b3-ba16-12f67dbe7d99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:29:53.720479 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3faf58-8784-40b3-ba16-12f67dbe7d99 admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:29:53.721194 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3faf58-8784-40b3-ba16-12f67dbe7d99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:29:53.722823 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3faf58-8784-40b3-ba16-12f67dbe7d99 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:29:53.732415 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1449/2875] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:29:53 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:29:53.737090 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e07aa4f-91bb-4be8-b0e5-4cdf80cd0cbd admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:29:53.747330 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e07aa4f-91bb-4be8-b0e5-4cdf80cd0cbd admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:29:53.752611 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd7b26d-1756-400c-b9a1-72d3a00d51e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:29:53.753507 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd7b26d-1756-400c-b9a1-72d3a00d51e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:29:53.753795 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd7b26d-1756-400c-b9a1-72d3a00d51e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:29:53.754068 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd7b26d-1756-400c-b9a1-72d3a00d51e6 admin admin] PATH_INFO: `/v3/projects/1c395e7385e1411ca7ae32e1b624d2e7/users/4ad59438688d42a9b5f64604ed88cc64/roles/21c65f2c51bd4dd283d13507d2714467` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:29:53.756035 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd7b26d-1756-400c-b9a1-72d3a00d51e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c395e7385e1411ca7ae32e1b624d2e7, user_id=4ad59438688d42a9b5f64604ed88cc64, role_id=21c65f2c51bd4dd283d13507d2714467)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:29:53.758725 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd7b26d-1756-400c-b9a1-72d3a00d51e6 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:29:53.768802 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1427/2876] 10.222.0.23 () {60 vars in 1482 bytes} [Sat Jun 18 05:29:53 2022] PUT /identity/v3/projects/1c395e7385e1411ca7ae32e1b624d2e7/users/4ad59438688d42a9b5f64604ed88cc64/roles/21c65f2c51bd4dd283d13507d2714467 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:29:53.778854 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92661ada-bf80-4edc-8d78-6abf7551b374 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:29:53.779148 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92661ada-bf80-4edc-8d78-6abf7551b374 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:29:53.779431 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92661ada-bf80-4edc-8d78-6abf7551b374 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:29:53.806561 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-92661ada-bf80-4edc-8d78-6abf7551b374 None None] MFA Rules not processed for user `4ad59438688d42a9b5f64604ed88cc64`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:29:53.807782 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-92661ada-bf80-4edc-8d78-6abf7551b374 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:29:53.875858 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1450/2877] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:29:53 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:29:53.886730 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd7b26d-1756-400c-b9a1-72d3a00d51e6 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:29:53.888408 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-fdd7b26d-1756-400c-b9a1-72d3a00d51e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:29:53.916834 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:29:53.946039 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd7b26d-1756-400c-b9a1-72d3a00d51e6 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:29:53.949668 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8079e68-08a3-4c23-98cc-98070aafa79c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9de0428abc8400aa338aba4f7c34c94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:29:53.950297 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8079e68-08a3-4c23-98cc-98070aafa79c service cinder] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:29:53.950580 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8079e68-08a3-4c23-98cc-98070aafa79c service cinder] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:29:53.950856 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8079e68-08a3-4c23-98cc-98070aafa79c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:29:54.006363 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8079e68-08a3-4c23-98cc-98070aafa79c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:29:54.007798 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8079e68-08a3-4c23-98cc-98070aafa79c service cinder] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:29:54.016804 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1428/2878] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:29:53 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:29:59.064282 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92661ada-bf80-4edc-8d78-6abf7551b374 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:29:59.065756 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-92661ada-bf80-4edc-8d78-6abf7551b374 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:29:59.078242 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92661ada-bf80-4edc-8d78-6abf7551b374 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:29:59.081828 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edab3171-3f2c-40ec-9428-ef75486d8230 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9de0428abc8400aa338aba4f7c34c94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:29:59.082456 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edab3171-3f2c-40ec-9428-ef75486d8230 service cinder] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:29:59.082737 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edab3171-3f2c-40ec-9428-ef75486d8230 service cinder] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:29:59.083009 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edab3171-3f2c-40ec-9428-ef75486d8230 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:29:59.092025 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edab3171-3f2c-40ec-9428-ef75486d8230 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:29:59.093691 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edab3171-3f2c-40ec-9428-ef75486d8230 service cinder] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:29:59.103370 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1451/2879] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:29:59 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:30:04.012874 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=37 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:30:04.814453 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8079e68-08a3-4c23-98cc-98070aafa79c service cinder] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:30:04.876850 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8079e68-08a3-4c23-98cc-98070aafa79c service cinder] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:30:04.881759 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384aeb9a-66ec-462f-bcc1-57c82712c61c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:30:04.882615 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384aeb9a-66ec-462f-bcc1-57c82712c61c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:30:04.882919 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384aeb9a-66ec-462f-bcc1-57c82712c61c admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:30:04.883202 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384aeb9a-66ec-462f-bcc1-57c82712c61c admin admin] PATH_INFO: `/v3/users/dae264d9829e4690894a764078ed7ebf` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:30:04.884388 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384aeb9a-66ec-462f-bcc1-57c82712c61c admin admin] RBAC: Authorizing `identity:delete_user(user_id=dae264d9829e4690894a764078ed7ebf)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:30:04.886366 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384aeb9a-66ec-462f-bcc1-57c82712c61c admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:30:05.068638 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-384aeb9a-66ec-462f-bcc1-57c82712c61c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dae264d9829e4690894a764078ed7ebf'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:30:05.069248 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-384aeb9a-66ec-462f-bcc1-57c82712c61c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dae264d9829e4690894a764078ed7ebf'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:30:05.132362 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-384aeb9a-66ec-462f-bcc1-57c82712c61c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dae264d9829e4690894a764078ed7ebf'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:30:05.145303 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-384aeb9a-66ec-462f-bcc1-57c82712c61c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dae264d9829e4690894a764078ed7ebf'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:30:05.152154 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-384aeb9a-66ec-462f-bcc1-57c82712c61c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dae264d9829e4690894a764078ed7ebf'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:30:05.159002 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1429/2880] 10.222.0.23 () {58 vars in 1223 bytes} [Sat Jun 18 05:30:04 2022] DELETE /identity/v3/users/dae264d9829e4690894a764078ed7ebf => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:30:05.163863 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15741a05-8f98-4834-8509-04f4461986f5 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:30:05.171489 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15741a05-8f98-4834-8509-04f4461986f5 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:30:05.171489 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15741a05-8f98-4834-8509-04f4461986f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:30:05.182339 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-15741a05-8f98-4834-8509-04f4461986f5 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:30:05.183536 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-15741a05-8f98-4834-8509-04f4461986f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:30:05.207842 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:30:05.243201 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1452/2881] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:30:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:30:05.253238 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384aeb9a-66ec-462f-bcc1-57c82712c61c admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:30:05.254687 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-384aeb9a-66ec-462f-bcc1-57c82712c61c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:30:05.311703 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384aeb9a-66ec-462f-bcc1-57c82712c61c admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:30:05.315523 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca469d7-6bed-4279-9194-a38b968b8c72 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c4d7cde8f4ad9aa479bb09e325db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:30:05.316154 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca469d7-6bed-4279-9194-a38b968b8c72 service neutron] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:30:05.316438 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca469d7-6bed-4279-9194-a38b968b8c72 service neutron] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:30:05.316715 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca469d7-6bed-4279-9194-a38b968b8c72 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:30:05.325971 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca469d7-6bed-4279-9194-a38b968b8c72 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:30:05.328077 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca469d7-6bed-4279-9194-a38b968b8c72 service neutron] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:30:05.337706 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1430/2882] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:30:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:30:05.735714 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15741a05-8f98-4834-8509-04f4461986f5 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:30:05.737467 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-15741a05-8f98-4834-8509-04f4461986f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:30:05.753709 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15741a05-8f98-4834-8509-04f4461986f5 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:30:05.760093 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361f7f5c-d6e2-4f54-accf-650b13e3cb89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:30:05.761516 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361f7f5c-d6e2-4f54-accf-650b13e3cb89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:30:05.761886 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361f7f5c-d6e2-4f54-accf-650b13e3cb89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:30:05.766954 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361f7f5c-d6e2-4f54-accf-650b13e3cb89 admin admin] PATH_INFO: `/v3/projects/3d4606866f934cfdb36bb9706914feb7` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:30:05.766954 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361f7f5c-d6e2-4f54-accf-650b13e3cb89 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d4606866f934cfdb36bb9706914feb7)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:30:05.766954 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361f7f5c-d6e2-4f54-accf-650b13e3cb89 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:30:05.897656 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-361f7f5c-d6e2-4f54-accf-650b13e3cb89 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d4606866f934cfdb36bb9706914feb7', 'request_id': 'req-361f7f5c-d6e2-4f54-accf-650b13e3cb89'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:30:05.903646 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-361f7f5c-d6e2-4f54-accf-650b13e3cb89 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d4606866f934cfdb36bb9706914feb7', 'request_id': 'req-361f7f5c-d6e2-4f54-accf-650b13e3cb89'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:30:05.910830 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-361f7f5c-d6e2-4f54-accf-650b13e3cb89 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d4606866f934cfdb36bb9706914feb7', 'request_id': 'req-361f7f5c-d6e2-4f54-accf-650b13e3cb89'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:30:06.015384 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-361f7f5c-d6e2-4f54-accf-650b13e3cb89 admin admin] The token cache is being invalidate because project 3d4606866f934cfdb36bb9706914feb7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80469) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 18 05:30:06.015947 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-361f7f5c-d6e2-4f54-accf-650b13e3cb89 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:30:06.016309 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-361f7f5c-d6e2-4f54-accf-650b13e3cb89 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:30:06.017885 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1453/2883] 10.222.0.23 () {58 vars in 1232 bytes} [Sat Jun 18 05:30:05 2022] DELETE /identity/v3/projects/3d4606866f934cfdb36bb9706914feb7 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:30:06.022036 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca469d7-6bed-4279-9194-a38b968b8c72 service neutron] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:30:06.033673 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca469d7-6bed-4279-9194-a38b968b8c72 service neutron] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:30:06.053233 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:30:06.100236 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4ff94c-2d5d-407d-9399-660bb774d4c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:30:06.102739 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4ff94c-2d5d-407d-9399-660bb774d4c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:30:06.103125 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4ff94c-2d5d-407d-9399-660bb774d4c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:30:06.103499 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4ff94c-2d5d-407d-9399-660bb774d4c3 admin admin] PATH_INFO: `/v3/users/4ad59438688d42a9b5f64604ed88cc64` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:30:06.104853 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4ff94c-2d5d-407d-9399-660bb774d4c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ad59438688d42a9b5f64604ed88cc64)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:30:06.106815 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4ff94c-2d5d-407d-9399-660bb774d4c3 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:30:06.278451 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-cb4ff94c-2d5d-407d-9399-660bb774d4c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ad59438688d42a9b5f64604ed88cc64'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:30:06.279109 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-cb4ff94c-2d5d-407d-9399-660bb774d4c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ad59438688d42a9b5f64604ed88cc64'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:30:06.332604 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-cb4ff94c-2d5d-407d-9399-660bb774d4c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ad59438688d42a9b5f64604ed88cc64'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:30:06.346549 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-cb4ff94c-2d5d-407d-9399-660bb774d4c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ad59438688d42a9b5f64604ed88cc64'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:30:06.353489 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-cb4ff94c-2d5d-407d-9399-660bb774d4c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ad59438688d42a9b5f64604ed88cc64'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:30:06.359650 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1431/2884] 10.222.0.23 () {58 vars in 1223 bytes} [Sat Jun 18 05:30:06 2022] DELETE /identity/v3/users/4ad59438688d42a9b5f64604ed88cc64 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:30:06.759472 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361f7f5c-d6e2-4f54-accf-650b13e3cb89 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:30:06.760946 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-361f7f5c-d6e2-4f54-accf-650b13e3cb89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:30:06.776608 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361f7f5c-d6e2-4f54-accf-650b13e3cb89 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:30:06.838828 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faa9a7d-7bb0-4dc6-8a93-3dba5fe57a46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:30:06.839645 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faa9a7d-7bb0-4dc6-8a93-3dba5fe57a46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:30:06.839965 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faa9a7d-7bb0-4dc6-8a93-3dba5fe57a46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:30:06.840195 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faa9a7d-7bb0-4dc6-8a93-3dba5fe57a46 admin admin] PATH_INFO: `/v3/projects/1c395e7385e1411ca7ae32e1b624d2e7` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:30:06.841400 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2faa9a7d-7bb0-4dc6-8a93-3dba5fe57a46 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c395e7385e1411ca7ae32e1b624d2e7)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:30:06.843483 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2faa9a7d-7bb0-4dc6-8a93-3dba5fe57a46 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:30:06.937217 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-2faa9a7d-7bb0-4dc6-8a93-3dba5fe57a46 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c395e7385e1411ca7ae32e1b624d2e7', 'request_id': 'req-2faa9a7d-7bb0-4dc6-8a93-3dba5fe57a46'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:30:06.941050 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:30:06.943208 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-2faa9a7d-7bb0-4dc6-8a93-3dba5fe57a46 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c395e7385e1411ca7ae32e1b624d2e7', 'request_id': 'req-2faa9a7d-7bb0-4dc6-8a93-3dba5fe57a46'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:30:06.949539 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-2faa9a7d-7bb0-4dc6-8a93-3dba5fe57a46 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c395e7385e1411ca7ae32e1b624d2e7', 'request_id': 'req-2faa9a7d-7bb0-4dc6-8a93-3dba5fe57a46'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:30:07.051611 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-2faa9a7d-7bb0-4dc6-8a93-3dba5fe57a46 admin admin] The token cache is being invalidate because project 1c395e7385e1411ca7ae32e1b624d2e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80469) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 18 05:30:07.051887 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-2faa9a7d-7bb0-4dc6-8a93-3dba5fe57a46 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:30:07.052508 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-2faa9a7d-7bb0-4dc6-8a93-3dba5fe57a46 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:30:07.054531 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1454/2885] 10.222.0.23 () {58 vars in 1232 bytes} [Sat Jun 18 05:30:06 2022] DELETE /identity/v3/projects/1c395e7385e1411ca7ae32e1b624d2e7 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:30:07.060646 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1c4cb5-fa77-4568-9952-a6ffddc192d1 None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:30:07.060943 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1c4cb5-fa77-4568-9952-a6ffddc192d1 None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:30:07.061229 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1c4cb5-fa77-4568-9952-a6ffddc192d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:30:07.079564 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-5d1c4cb5-fa77-4568-9952-a6ffddc192d1 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:30:07.080801 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-5d1c4cb5-fa77-4568-9952-a6ffddc192d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:30:07.141263 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1432/2886] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:30:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:30:07.149362 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99cfb6b-40b0-4894-be2b-86dd6fb71718 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:30:07.149682 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99cfb6b-40b0-4894-be2b-86dd6fb71718 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:30:07.149962 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99cfb6b-40b0-4894-be2b-86dd6fb71718 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:30:07.169006 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-f99cfb6b-40b0-4894-be2b-86dd6fb71718 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:30:07.170287 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-f99cfb6b-40b0-4894-be2b-86dd6fb71718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:30:07.178600 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1455/2887] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:30:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:30:07.182739 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1c4cb5-fa77-4568-9952-a6ffddc192d1 None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:30:07.192588 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1c4cb5-fa77-4568-9952-a6ffddc192d1 None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:30:07.197166 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380d718f-4cbf-436c-86ed-816bff603945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:30:07.197847 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380d718f-4cbf-436c-86ed-816bff603945 admin admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:30:07.198129 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380d718f-4cbf-436c-86ed-816bff603945 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:30:07.198404 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380d718f-4cbf-436c-86ed-816bff603945 admin admin] PATH_INFO: `/v3/domains` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:30:07.199114 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380d718f-4cbf-436c-86ed-816bff603945 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80468) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 18 05:30:07.199420 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380d718f-4cbf-436c-86ed-816bff603945 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:30:07.201005 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380d718f-4cbf-436c-86ed-816bff603945 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:30:07.213489 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1433/2888] 10.222.0.23 () {58 vars in 1152 bytes} [Sat Jun 18 05:30:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 18 05:30:07.218396 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99cfb6b-40b0-4894-be2b-86dd6fb71718 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:30:07.227891 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99cfb6b-40b0-4894-be2b-86dd6fb71718 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:30:07.232535 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d51dda1-857a-4b2a-a22b-719faa5e8030 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:30:07.233951 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d51dda1-857a-4b2a-a22b-719faa5e8030 admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:30:07.234278 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d51dda1-857a-4b2a-a22b-719faa5e8030 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:30:07.234554 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d51dda1-857a-4b2a-a22b-719faa5e8030 admin admin] PATH_INFO: `/v3/projects` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:30:07.235252 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d51dda1-857a-4b2a-a22b-719faa5e8030 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:30:07.237150 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d51dda1-857a-4b2a-a22b-719faa5e8030 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:30:07.310352 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1456/2889] 10.222.0.23 () {60 vars in 1152 bytes} [Sat Jun 18 05:30:07 2022] POST /identity/v3/projects => generated 383 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:30:07.314407 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380d718f-4cbf-436c-86ed-816bff603945 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:30:07.324459 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380d718f-4cbf-436c-86ed-816bff603945 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:30:07.352304 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:30:07.383108 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897606a5-8d1d-4be6-923d-687bf68c4cc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:30:07.383785 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897606a5-8d1d-4be6-923d-687bf68c4cc0 admin admin] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:30:07.384052 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897606a5-8d1d-4be6-923d-687bf68c4cc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:30:07.384329 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897606a5-8d1d-4be6-923d-687bf68c4cc0 admin admin] PATH_INFO: `/v3/users` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:30:07.385026 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897606a5-8d1d-4be6-923d-687bf68c4cc0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:30:07.387018 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897606a5-8d1d-4be6-923d-687bf68c4cc0 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:30:07.481022 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1434/2890] 10.222.0.23 () {60 vars in 1143 bytes} [Sat Jun 18 05:30:07 2022] POST /identity/v3/users => generated 292 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 18 05:30:07.485899 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d51dda1-857a-4b2a-a22b-719faa5e8030 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:30:07.496379 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d51dda1-857a-4b2a-a22b-719faa5e8030 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:30:07.501751 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f8c3e1-23e2-40ba-b5dc-8134d77a6921 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:30:07.502430 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f8c3e1-23e2-40ba-b5dc-8134d77a6921 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:30:07.502710 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f8c3e1-23e2-40ba-b5dc-8134d77a6921 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:30:07.502982 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f8c3e1-23e2-40ba-b5dc-8134d77a6921 admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:30:07.504346 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f8c3e1-23e2-40ba-b5dc-8134d77a6921 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:30:07.505455 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f8c3e1-23e2-40ba-b5dc-8134d77a6921 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:30:07.516676 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1457/2891] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:30:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:30:07.520557 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897606a5-8d1d-4be6-923d-687bf68c4cc0 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:30:07.532223 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897606a5-8d1d-4be6-923d-687bf68c4cc0 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:30:07.537535 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4afa9b-268b-4d28-9f66-c22b03da7aef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:30:07.538800 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4afa9b-268b-4d28-9f66-c22b03da7aef admin admin] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:30:07.538800 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4afa9b-268b-4d28-9f66-c22b03da7aef admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:30:07.538800 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4afa9b-268b-4d28-9f66-c22b03da7aef admin admin] PATH_INFO: `/v3/roles` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:30:07.539616 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4afa9b-268b-4d28-9f66-c22b03da7aef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:30:07.541243 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4afa9b-268b-4d28-9f66-c22b03da7aef admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:30:07.552222 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1435/2892] 10.222.0.23 () {58 vars in 1121 bytes} [Sat Jun 18 05:30:07 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 18 05:30:07.556398 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f8c3e1-23e2-40ba-b5dc-8134d77a6921 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:30:07.566666 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f8c3e1-23e2-40ba-b5dc-8134d77a6921 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:30:07.571338 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b972c1a6-b94d-4248-b021-fdb7689352f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:30:07.572206 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b972c1a6-b94d-4248-b021-fdb7689352f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:30:07.572486 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b972c1a6-b94d-4248-b021-fdb7689352f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:30:07.572759 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b972c1a6-b94d-4248-b021-fdb7689352f9 admin admin] PATH_INFO: `/v3/projects/74bfbdabdbca4b1ab62e2327f3ee8e56/users/99a4acebd642457fbb8be4ad320477ee/roles/cbf5c4c26898413d93e73b46b41046a3` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:30:07.587411 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b972c1a6-b94d-4248-b021-fdb7689352f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=74bfbdabdbca4b1ab62e2327f3ee8e56, user_id=99a4acebd642457fbb8be4ad320477ee, role_id=cbf5c4c26898413d93e73b46b41046a3)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:30:07.590102 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b972c1a6-b94d-4248-b021-fdb7689352f9 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:30:07.681918 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1458/2893] 10.222.0.23 () {60 vars in 1482 bytes} [Sat Jun 18 05:30:07 2022] PUT /identity/v3/projects/74bfbdabdbca4b1ab62e2327f3ee8e56/users/99a4acebd642457fbb8be4ad320477ee/roles/cbf5c4c26898413d93e73b46b41046a3 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:30:07.694699 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b06ae12-6e3e-4c21-82e4-9b922380455b None None] REQUEST_METHOD: `POST` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:30:07.694984 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b06ae12-6e3e-4c21-82e4-9b922380455b None None] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:30:07.695260 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b06ae12-6e3e-4c21-82e4-9b922380455b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:30:07.722916 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.auth.core [None req-3b06ae12-6e3e-4c21-82e4-9b922380455b None None] MFA Rules not processed for user `99a4acebd642457fbb8be4ad320477ee`. Rule list: `[]` (Enabled: `True`). {{(pid=80468) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:30:07.724146 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-3b06ae12-6e3e-4c21-82e4-9b922380455b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:30:07.788425 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1436/2894] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:30:07 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:30:07.799829 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b972c1a6-b94d-4248-b021-fdb7689352f9 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:30:07.801695 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-b972c1a6-b94d-4248-b021-fdb7689352f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:30:07.858124 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b972c1a6-b94d-4248-b021-fdb7689352f9 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:30:07.861804 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f576a3ca-ebb0-4501-8714-ca67c554511a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9de0428abc8400aa338aba4f7c34c94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:30:07.862473 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f576a3ca-ebb0-4501-8714-ca67c554511a service cinder] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:30:07.862756 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f576a3ca-ebb0-4501-8714-ca67c554511a service cinder] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:30:07.863035 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f576a3ca-ebb0-4501-8714-ca67c554511a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:30:07.871341 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f576a3ca-ebb0-4501-8714-ca67c554511a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:30:07.872784 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f576a3ca-ebb0-4501-8714-ca67c554511a service cinder] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:30:07.881717 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1459/2895] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:30:07 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:30:17.785533 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=53,keystone:INSERT=3 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:30:17.877638 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:DELETE=2,keystone:SELECT=58,keystone:INSERT=3 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:30:17.880276 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b06ae12-6e3e-4c21-82e4-9b922380455b None None] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:30:17.943723 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b06ae12-6e3e-4c21-82e4-9b922380455b None None] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:30:17.948365 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ce3342-18cb-436a-9f08-3a9bd127d398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:30:17.949219 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ce3342-18cb-436a-9f08-3a9bd127d398 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:30:17.949614 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ce3342-18cb-436a-9f08-3a9bd127d398 admin admin] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:30:17.949896 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ce3342-18cb-436a-9f08-3a9bd127d398 admin admin] PATH_INFO: `/v3/users/99a4acebd642457fbb8be4ad320477ee` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:30:17.951138 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ce3342-18cb-436a-9f08-3a9bd127d398 admin admin] RBAC: Authorizing `identity:delete_user(user_id=99a4acebd642457fbb8be4ad320477ee)` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:30:17.953806 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ce3342-18cb-436a-9f08-3a9bd127d398 admin admin] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:30:18.166352 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-34ce3342-18cb-436a-9f08-3a9bd127d398 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '99a4acebd642457fbb8be4ad320477ee'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:30:18.166972 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-34ce3342-18cb-436a-9f08-3a9bd127d398 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '99a4acebd642457fbb8be4ad320477ee'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:30:18.241057 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-34ce3342-18cb-436a-9f08-3a9bd127d398 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '99a4acebd642457fbb8be4ad320477ee'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:30:18.254000 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-34ce3342-18cb-436a-9f08-3a9bd127d398 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '99a4acebd642457fbb8be4ad320477ee'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:30:18.260792 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.notifications [None req-34ce3342-18cb-436a-9f08-3a9bd127d398 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '99a4acebd642457fbb8be4ad320477ee'} {{(pid=80468) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:30:18.266915 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1437/2896] 10.222.0.23 () {58 vars in 1223 bytes} [Sat Jun 18 05:30:17 2022] DELETE /identity/v3/users/99a4acebd642457fbb8be4ad320477ee => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:30:18.271625 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a471c06-cb59-4a58-bca6-d0bc32c2fd29 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:30:18.271943 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a471c06-cb59-4a58-bca6-d0bc32c2fd29 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:30:18.272329 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a471c06-cb59-4a58-bca6-d0bc32c2fd29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:30:18.289707 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-7a471c06-cb59-4a58-bca6-d0bc32c2fd29 None None] MFA Rules not processed for user `5d3e83f3ca7347dea1350fa29bac5d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 18 05:30:18.290901 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-7a471c06-cb59-4a58-bca6-d0bc32c2fd29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:30:18.348385 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1460/2897] 10.222.0.23 () {58 vars in 957 bytes} [Sat Jun 18 05:30:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 18 05:30:18.358637 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ce3342-18cb-436a-9f08-3a9bd127d398 admin admin] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:30:18.360064 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-34ce3342-18cb-436a-9f08-3a9bd127d398 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:30:18.418328 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ce3342-18cb-436a-9f08-3a9bd127d398 admin admin] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:30:18.422367 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abf8ec9-8553-4a34-800e-e57a965e1be0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c4d7cde8f4ad9aa479bb09e325db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:30:18.423000 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abf8ec9-8553-4a34-800e-e57a965e1be0 service neutron] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:30:18.423283 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abf8ec9-8553-4a34-800e-e57a965e1be0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:30:18.423557 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abf8ec9-8553-4a34-800e-e57a965e1be0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:30:18.432532 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abf8ec9-8553-4a34-800e-e57a965e1be0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:30:18.433993 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abf8ec9-8553-4a34-800e-e57a965e1be0 service neutron] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:30:18.443883 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1438/2898] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:30:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:30:18.873690 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a471c06-cb59-4a58-bca6-d0bc32c2fd29 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:30:18.877373 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-7a471c06-cb59-4a58-bca6-d0bc32c2fd29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:30:18.887527 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a471c06-cb59-4a58-bca6-d0bc32c2fd29 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:30:18.893450 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be585d8-adfb-4583-8bbd-67cb9bfc1b84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d3e83f3ca7347dea1350fa29bac5d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd587d7347f6f4de2882df28dc6b6a574', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:30:18.894327 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be585d8-adfb-4583-8bbd-67cb9bfc1b84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:30:18.894649 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be585d8-adfb-4583-8bbd-67cb9bfc1b84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:30:18.894890 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be585d8-adfb-4583-8bbd-67cb9bfc1b84 admin admin] PATH_INFO: `/v3/projects/74bfbdabdbca4b1ab62e2327f3ee8e56` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:30:18.896119 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be585d8-adfb-4583-8bbd-67cb9bfc1b84 admin admin] RBAC: Authorizing `identity:delete_project(project_id=74bfbdabdbca4b1ab62e2327f3ee8e56)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:30:18.898084 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be585d8-adfb-4583-8bbd-67cb9bfc1b84 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:30:19.038559 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-5be585d8-adfb-4583-8bbd-67cb9bfc1b84 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '74bfbdabdbca4b1ab62e2327f3ee8e56', 'request_id': 'req-5be585d8-adfb-4583-8bbd-67cb9bfc1b84'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:30:19.044266 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-5be585d8-adfb-4583-8bbd-67cb9bfc1b84 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '74bfbdabdbca4b1ab62e2327f3ee8e56', 'request_id': 'req-5be585d8-adfb-4583-8bbd-67cb9bfc1b84'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:30:19.050699 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-5be585d8-adfb-4583-8bbd-67cb9bfc1b84 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '74bfbdabdbca4b1ab62e2327f3ee8e56', 'request_id': 'req-5be585d8-adfb-4583-8bbd-67cb9bfc1b84'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:30:19.122651 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-5be585d8-adfb-4583-8bbd-67cb9bfc1b84 admin admin] The token cache is being invalidate because project 74bfbdabdbca4b1ab62e2327f3ee8e56 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80469) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 18 05:30:19.122940 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-5be585d8-adfb-4583-8bbd-67cb9bfc1b84 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:30:19.123587 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-5be585d8-adfb-4583-8bbd-67cb9bfc1b84 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 18 05:30:19.125264 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1461/2899] 10.222.0.23 () {58 vars in 1232 bytes} [Sat Jun 18 05:30:18 2022] DELETE /identity/v3/projects/74bfbdabdbca4b1ab62e2327f3ee8e56 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 18 05:30:28.439563 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:30:29.119506 dv-cinder-836996-35-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 18 05:30:47.433583 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abf8ec9-8553-4a34-800e-e57a965e1be0 service neutron] Authenticating user token {{(pid=80468) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 18 05:30:47.434965 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-9abf8ec9-8553-4a34-800e-e57a965e1be0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:30:47.522971 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abf8ec9-8553-4a34-800e-e57a965e1be0 service neutron] Validating token access rules against request {{(pid=80468) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 18 05:30:47.582918 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d689c0b5-8abf-4924-9983-2c5f8be3e00a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5323ffd91f864680b0c0084eb3f44d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c95f8f3084c5472cb8364b7c57207394', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80468) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 18 05:30:47.583571 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d689c0b5-8abf-4924-9983-2c5f8be3e00a service placement] REQUEST_METHOD: `GET` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 18 05:30:47.583857 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d689c0b5-8abf-4924-9983-2c5f8be3e00a service placement] SCRIPT_NAME: `/identity` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 18 05:30:47.584134 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d689c0b5-8abf-4924-9983-2c5f8be3e00a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80468) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 18 05:30:47.585982 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.fernet_utils [None req-d689c0b5-8abf-4924-9983-2c5f8be3e00a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80468) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 18 05:30:47.597756 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d689c0b5-8abf-4924-9983-2c5f8be3e00a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 18 05:30:47.599400 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d689c0b5-8abf-4924-9983-2c5f8be3e00a service placement] RBAC: Authorization granted {{(pid=80468) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 18 05:30:47.609439 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: [pid: 80468|app: 0|req: 1439/2900] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Jun 18 05:30:47 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 18 05:30:57.605023 dv-cinder-836996-35-iscsi devstack@keystone.service[80468]: DEBUG dbcounter [-] [80468] Writing DB stats keystone:SELECT=67 {{(pid=80468) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}